2024-09-05T13:04:22.250 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-05T13:04:22.266 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T13:04:22.343 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478 branch: wip-bharath4-testing-2024-09-02-0957 description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-bitmap} rados supported-random-distro$/{centos_latest}} email: skanta@redhat.com first_in_suite: false job_id: '7890478' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath4-testing-2024-09-02-0957 ceph: conf: global: mon election default strategy: 3 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2: false bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) mon_bind_msgr2: false sha1: ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath4-testing-2024-09-02-0957 sha1: ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 5171 sha1: ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath4-testing-2024-09-02-0957 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03 targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNFW74Bw72hrhPkH0GuhcO/JNalMqdIhZBuVS0J+H97uL9ZLUloQvoNDA/+pcgGNqxXvUKMaMFqmPIAiHl/iV9c= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - rep_lost_unfound_delete: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-05_12:54:40 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-05T13:04:22.343 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/qa; will attempt to use it 2024-09-05T13:04:22.344 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/qa/tasks 2024-09-05T13:04:22.344 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-05T13:04:22.346 INFO:teuthology.task.internal:Checking packages... 2024-09-05T13:04:22.366 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03' 2024-09-05T13:04:22.366 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-05T13:04:22.366 INFO:teuthology.packaging:ref: None 2024-09-05T13:04:22.366 INFO:teuthology.packaging:tag: None 2024-09-05T13:04:22.366 INFO:teuthology.packaging:branch: wip-bharath4-testing-2024-09-02-0957 2024-09-05T13:04:22.366 INFO:teuthology.packaging:sha1: ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03 2024-09-05T13:04:22.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath4-testing-2024-09-02-0957 2024-09-05T13:04:22.505 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4650.gac9df2d6 2024-09-05T13:04:22.506 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-05T13:04:22.512 INFO:teuthology.task.internal:no buildpackages task found 2024-09-05T13:04:22.513 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-05T13:04:22.518 INFO:teuthology.task.internal:Saving configuration 2024-09-05T13:04:22.530 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-05T13:04:22.537 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-05T13:04:22.553 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 12:59:33.892948', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNFW74Bw72hrhPkH0GuhcO/JNalMqdIhZBuVS0J+H97uL9ZLUloQvoNDA/+pcgGNqxXvUKMaMFqmPIAiHl/iV9c='} 2024-09-05T13:04:22.553 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-05T13:04:22.558 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-05T13:04:22.558 INFO:teuthology.run_tasks:Running task console_log... 2024-09-05T13:04:22.583 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2a5207fbe0>, signals=[15]) 2024-09-05T13:04:22.583 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-05T13:04:22.589 INFO:teuthology.task.internal:Opening connections... 2024-09-05T13:04:22.589 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:04:22.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T13:04:22.667 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-05T13:04:22.673 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-09-05T13:04:22.746 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-09-05T13:04:22.747 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T13:04:22.805 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-09-05T13:04:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T13:04:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T13:04:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T13:04:22.806 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-09-05T13:04:22.830 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-05T13:04:22.838 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-05T13:04:22.843 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-05T13:04:22.843 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-09-05T13:04:22.862 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-05T13:04:22.867 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-05T13:04:22.867 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-09-05T13:04:23.007 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-05T13:04:23.036 INFO:teuthology.run_tasks:Running task kernel... 2024-09-05T13:04:23.050 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-05T13:04:23.050 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-05T13:04:23.052 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T13:04:23.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:04:23.070 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-09-05T13:04:23.127 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-503.el9.x86_64 2024-09-05T13:04:23.128 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.14.0-503.el9.x86_64 2024-09-05T13:04:23.128 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum install -y kernel 2024-09-05T13:04:27.916 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.2 MB 00:03 2024-09-05T13:04:30.584 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 13 MB/s | 20 MB 00:01 2024-09-05T13:04:36.834 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-05T13:04:38.493 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-05T13:04:39.272 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-05T13:04:44.720 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-05T13:04:46.701 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T13:04:46.701 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T13:04:46.784 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:04:46.785 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:04:46.785 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:04:46.915 DEBUG:teuthology.orchestra.run.smithi086:> echo no | sudo yum reinstall kernel || true 2024-09-05T13:04:47.451 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 01:04:44 PM UTC. 2024-09-05T13:04:47.606 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:04:47.618 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:04:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-05T13:04:47.618 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:04:47.618 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalling: 2024-09-05T13:04:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T13:04:47.618 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:04:47.618 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:04:47.618 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:04:47.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:04:47.619 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 1.9 M 2024-09-05T13:04:47.619 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 0 2024-09-05T13:04:47.619 INFO:teuthology.orchestra.run.smithi086.stderr:Operation aborted. 2024-09-05T13:04:47.679 INFO:teuthology.orchestra.run.smithi086.stdout:Is this ok [y/N]: 2024-09-05T13:04:47.679 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum reinstall -y kernel || true 2024-09-05T13:04:48.211 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:04 ago on Thu 05 Sep 2024 01:04:44 PM UTC. 2024-09-05T13:04:48.337 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalling: 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:04:48.338 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 1.9 M 2024-09-05T13:04:48.339 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 0 2024-09-05T13:04:48.339 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:04:49.314 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-503.el9.x86_64.rpm 2.0 MB/s | 1.9 MB 00:00 2024-09-05T13:04:49.314 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:04:49.314 INFO:teuthology.orchestra.run.smithi086.stdout:Total 1.9 MB/s | 1.9 MB 00:00 2024-09-05T13:04:49.343 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:04:49.582 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:04:49.582 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:04:49.612 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:04:49.612 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:04:49.771 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:04:49.882 INFO:teuthology.orchestra.run.smithi086.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T13:04:49.883 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T13:04:50.582 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T13:04:50.582 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T13:04:51.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T13:04:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:04:51.334 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalled: 2024-09-05T13:04:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T13:04:51.334 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:04:51.334 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:04:51.518 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T13:04:51.577 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T13:04:51.577 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T13:04:51.577 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T13:04:51.578 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-05T13:04:51.578 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T13:04:51.611 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2024-09-05T13:04:51.636 DEBUG:teuthology.parallel:result is None 2024-09-05T13:04:51.636 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-05T13:04:51.645 INFO:teuthology.task.internal:Creating test directory... 2024-09-05T13:04:51.646 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T13:04:51.692 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-05T13:04:51.698 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-05T13:04:51.704 INFO:teuthology.task.internal:Creating archive directory... 2024-09-05T13:04:51.704 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T13:04:51.756 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-05T13:04:51.763 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-05T13:04:51.763 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T13:04:51.825 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T13:04:51.835 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T13:04:51.836 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-05T13:04:51.843 INFO:teuthology.task.internal:Configuring sudo... 2024-09-05T13:04:51.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T13:04:51.901 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-05T13:04:51.910 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-05T13:04:51.910 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T13:04:51.957 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T13:04:52.031 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T13:04:52.086 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T13:04:52.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T13:04:52.208 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:04:52.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T13:04:52.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-09-05T13:04:52.350 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T13:04:52.755 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-05T13:04:52.763 INFO:teuthology.task.internal:Starting timer... 2024-09-05T13:04:52.763 INFO:teuthology.run_tasks:Running task pcp... 2024-09-05T13:04:52.793 INFO:teuthology.run_tasks:Running task selinux... 2024-09-05T13:04:52.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-09-05T13:04:52.837 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-09-05T13:04:52.839 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-05T13:04:52.839 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-09-05T13:04:52.899 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-09-05T13:04:52.899 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive'} 2024-09-05T13:04:52.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-05T13:04:52.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:04:52.964 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-05T13:04:52.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-09-05T13:04:53.036 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-05T13:04:53.045 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-05T13:04:53.056 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-05T13:04:53.057 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-05T13:08:30.589 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com')] 2024-09-05T13:08:30.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-05T13:08:30.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T13:08:30.722 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-09-05T13:08:30.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-09-05T13:08:30.800 INFO:teuthology.run_tasks:Running task clock... 2024-09-05T13:08:30.810 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-05T13:08:30.810 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T13:08:30.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T13:08:30.868 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T13:08:30.882 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T13:08:30.909 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-09-05T13:08:30.917 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-09-05T13:08:30.932 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T13:08:30.945 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T13:08:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-09-05T13:08:31.008 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T13:08:31.008 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-09-05T13:08:31.008 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T13:08:31.008 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T13:08:31.008 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T13:08:31.008 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T13:08:31.009 INFO:teuthology.run_tasks:Running task install... 2024-09-05T13:08:31.017 DEBUG:teuthology.task.install:project ceph 2024-09-05T13:08:31.017 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03'}} 2024-09-05T13:08:31.018 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03'} 2024-09-05T13:08:31.018 INFO:teuthology.task.install:Using flavor: default 2024-09-05T13:08:31.024 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-05T13:08:31.024 INFO:teuthology.task.install:extra packages: [] 2024-09-05T13:08:31.024 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03', 'tag': None, 'wait_for_package': False} 2024-09-05T13:08:31.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03 2024-09-05T13:08:31.170 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath4-testing-2024-09-02-0957/ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/centos/9/flavors/default/ 2024-09-05T13:08:31.170 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4650.gac9df2d6 2024-09-05T13:08:31.284 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath4-testing-2024-09-02-0957/ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath4-testing-2024-09-02-0957/ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath4-testing-2024-09-02-0957/ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T13:08:31.284 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:08:31.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T13:08:31.310 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-05T13:08:31.311 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T13:08:31.377 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T13:08:31.454 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T13:08:31.522 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-09-05T13:08:31.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-09-05T13:08:31.745 INFO:teuthology.orchestra.run.smithi086.stdout:45 files removed 2024-09-05T13:08:31.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw 2024-09-05T13:08:32.415 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 227 kB/s | 84 kB 00:00 2024-09-05T13:08:32.807 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 69 kB/s | 24 kB 00:00 2024-09-05T13:08:33.099 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-05T13:08:33.824 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-05T13:08:40.742 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 20 MB 00:05 2024-09-05T13:08:46.026 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-05T13:08:47.435 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-05T13:08:48.281 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-05T13:08:53.678 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-05T13:08:55.912 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 11 M 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 3.5 M 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 3.2 M 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 5.5 M 2024-09-05T13:08:55.915 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 22 M 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 25 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 755 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 509 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 5.5 M 2024-09-05T13:08:55.916 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 45 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 143 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 161 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 320 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 306 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 99 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T13:08:55.917 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-09-05T13:08:55.918 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T13:08:55.918 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:08:55.918 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:08:55.918 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:08:55.918 INFO:teuthology.orchestra.run.smithi086.stdout:Install 30 Packages 2024-09-05T13:08:55.918 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 2 Packages 2024-09-05T13:08:55.918 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:08:55.918 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 62 M 2024-09-05T13:08:55.919 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:08:57.562 INFO:teuthology.orchestra.run.smithi086.stdout:(1/32): ceph-base-19.3.0-4650.gac9df2d6.el9.x86 4.4 MB/s | 5.5 MB 00:01 2024-09-05T13:08:57.663 INFO:teuthology.orchestra.run.smithi086.stdout:(2/32): ceph-selinux-19.3.0-4650.gac9df2d6.el9. 252 kB/s | 25 kB 00:00 2024-09-05T13:08:57.904 INFO:teuthology.orchestra.run.smithi086.stdout:(3/32): libcephfs2-19.3.0-4650.gac9df2d6.el9.x8 3.1 MB/s | 755 kB 00:00 2024-09-05T13:08:58.021 INFO:teuthology.orchestra.run.smithi086.stdout:(4/32): ceph-radosgw-19.3.0-4650.gac9df2d6.el9. 6.4 MB/s | 11 MB 00:01 2024-09-05T13:08:58.063 INFO:teuthology.orchestra.run.smithi086.stdout:(5/32): libradosstriper1-19.3.0-4650.gac9df2d6. 3.1 MB/s | 509 kB 00:00 2024-09-05T13:08:58.289 INFO:teuthology.orchestra.run.smithi086.stdout:(6/32): ceph-common-19.3.0-4650.gac9df2d6.el9.x 11 MB/s | 22 MB 00:01 2024-09-05T13:08:58.314 INFO:teuthology.orchestra.run.smithi086.stdout:(7/32): python3-ceph-argparse-19.3.0-4650.gac9d 180 kB/s | 45 kB 00:00 2024-09-05T13:08:58.356 INFO:teuthology.orchestra.run.smithi086.stdout:(8/32): python3-ceph-common-19.3.0-4650.gac9df2 2.1 MB/s | 143 kB 00:00 2024-09-05T13:08:58.429 INFO:teuthology.orchestra.run.smithi086.stdout:(9/32): python3-cephfs-19.3.0-4650.gac9df2d6.el 1.4 MB/s | 161 kB 00:00 2024-09-05T13:08:58.473 INFO:teuthology.orchestra.run.smithi086.stdout:(10/32): python3-rados-19.3.0-4650.gac9df2d6.el 2.7 MB/s | 320 kB 00:00 2024-09-05T13:08:58.498 INFO:teuthology.orchestra.run.smithi086.stdout:(11/32): python3-rbd-19.3.0-4650.gac9df2d6.el9. 4.3 MB/s | 306 kB 00:00 2024-09-05T13:08:58.590 INFO:teuthology.orchestra.run.smithi086.stdout:(12/32): librgw2-19.3.0-4650.gac9df2d6.el9.x86_ 9.7 MB/s | 5.5 MB 00:00 2024-09-05T13:08:58.615 INFO:teuthology.orchestra.run.smithi086.stdout:(13/32): python3-rgw-19.3.0-4650.gac9df2d6.el9. 701 kB/s | 99 kB 00:00 2024-09-05T13:08:58.690 INFO:teuthology.orchestra.run.smithi086.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 556 kB/s | 107 kB 00:00 2024-09-05T13:08:58.724 INFO:teuthology.orchestra.run.smithi086.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-05T13:08:58.749 INFO:teuthology.orchestra.run.smithi086.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-05T13:08:58.791 INFO:teuthology.orchestra.run.smithi086.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-05T13:08:58.816 INFO:teuthology.orchestra.run.smithi086.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-09-05T13:08:58.842 INFO:teuthology.orchestra.run.smithi086.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-05T13:08:58.859 INFO:teuthology.orchestra.run.smithi086.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 858 kB/s | 36 kB 00:00 2024-09-05T13:08:58.884 INFO:teuthology.orchestra.run.smithi086.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-05T13:08:58.901 INFO:teuthology.orchestra.run.smithi086.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-09-05T13:08:58.959 INFO:teuthology.orchestra.run.smithi086.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.0 MB/s | 309 kB 00:00 2024-09-05T13:08:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2024-09-05T13:08:58.993 INFO:teuthology.orchestra.run.smithi086.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 337 kB/s | 26 kB 00:00 2024-09-05T13:08:59.010 INFO:teuthology.orchestra.run.smithi086.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-05T13:08:59.035 INFO:teuthology.orchestra.run.smithi086.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-05T13:08:59.086 INFO:teuthology.orchestra.run.smithi086.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-05T13:08:59.144 INFO:teuthology.orchestra.run.smithi086.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-05T13:08:59.211 INFO:teuthology.orchestra.run.smithi086.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-05T13:08:59.353 INFO:teuthology.orchestra.run.smithi086.stdout:(31/32): librbd1-19.3.0-4650.gac9df2d6.el9.x86_ 15 MB/s | 3.2 MB 00:00 2024-09-05T13:08:59.437 INFO:teuthology.orchestra.run.smithi086.stdout:(32/32): librados2-19.3.0-4650.gac9df2d6.el9.x8 9.9 MB/s | 3.5 MB 00:00 2024-09-05T13:08:59.442 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:08:59.442 INFO:teuthology.orchestra.run.smithi086.stdout:Total 17 MB/s | 62 MB 00:03 2024-09-05T13:08:59.545 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:08:59.581 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:08:59.582 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:09:00.056 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:09:00.056 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:09:01.498 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:09:01.572 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-05T13:09:01.599 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-05T13:09:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-05T13:09:01.656 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-05T13:09:01.683 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-05T13:09:01.891 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-05T13:09:01.943 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-05T13:09:02.324 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-05T13:09:02.385 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.3.0-4650.gac9df2d6.el9.x86_64 7/34 2024-09-05T13:09:02.479 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.3.0-4650.gac9df2d6.el9.x86_64 7/34 2024-09-05T13:09:02.567 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.3.0-4650.gac9df2d6.el9.x86_64 8/34 2024-09-05T13:09:02.595 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.3.0-4650.gac9df2d6.el9.x86_64 9/34 2024-09-05T13:09:02.649 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.3.0-4650.gac9df2d6.el9.x86_64 9/34 2024-09-05T13:09:02.682 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-05T13:09:02.730 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.3.0-4650.gac9df2d6.el9. 11/34 2024-09-05T13:09:02.800 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.3.0-4650.gac9df2d6.el9.x86_64 12/34 2024-09-05T13:09:02.822 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.3.0-4650.gac9df2d6.el9.x86_6 13/34 2024-09-05T13:09:02.890 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4650.gac9df2d6.el9.x86_6 13/34 2024-09-05T13:09:02.924 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-05T13:09:03.079 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-05T13:09:03.112 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-05T13:09:03.152 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-05T13:09:03.178 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-09-05T13:09:03.206 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-05T13:09:03.240 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-05T13:09:03.559 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-05T13:09:03.601 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.3.0-4650.gac9df2d6.el9.x86_64 22/34 2024-09-05T13:09:03.691 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.3.0-4650.gac9df2d6.el9.x86_64 22/34 2024-09-05T13:09:03.721 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.3.0-4650.gac9df2d6.el9.x86_64 23/34 2024-09-05T13:09:03.757 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-05T13:09:04.246 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-05T13:09:04.272 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.3.0-4650.gac9df2d6.el9.x86_64 26/34 2024-09-05T13:09:04.332 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.3.0-4650.gac9df2d6.el9.x86_64 26/34 2024-09-05T13:09:04.403 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.3.0-4650.gac9df2d6.el9.x86_64 27/34 2024-09-05T13:09:04.439 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.3.0-4650.gac9df2d6.el9.x8 28/34 2024-09-05T13:09:06.721 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.3.0-4650.gac9df2d6.el9.x86_64 29/34 2024-09-05T13:09:06.760 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.3.0-4650.gac9df2d6.el9.x86_64 29/34 2024-09-05T13:09:07.313 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.3.0-4650.gac9df2d6.el9.x86_64 29/34 2024-09-05T13:09:07.363 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.3.0-4650.gac9df2d6.el9.x86_64 30/34 2024-09-05T13:09:07.447 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.3.0-4650.gac9df2d6.el9.x86_64 30/34 2024-09-05T13:09:07.448 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T13:09:07.448 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T13:09:07.448 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:07.481 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.3.0-4650.gac9df2d6.el9.x86_64 31/34 2024-09-05T13:09:17.388 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4650.gac9df2d6.el9.x86_64 31/34 2024-09-05T13:09:17.389 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-09-05T13:09:17.389 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-09-05T13:09:17.389 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-09-05T13:09:17.389 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-09-05T13:09:17.389 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-09-05T13:09:17.389 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-09-05T13:09:17.389 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-09-05T13:09:17.389 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:18.352 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.3.0-4650.gac9df2d6.el9.x86_64 32/34 2024-09-05T13:09:18.377 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4650.gac9df2d6.el9.x86_64 32/34 2024-09-05T13:09:18.377 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T13:09:18.377 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T13:09:18.377 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T13:09:18.378 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T13:09:18.378 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:18.379 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-05T13:09:18.471 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-05T13:09:18.471 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-05T13:09:19.613 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-05T13:09:19.613 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/34 2024-09-05T13:09:19.614 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.3.0-4650.gac9df2d6.el9.x86_64 2/34 2024-09-05T13:09:19.614 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.3.0-4650.gac9df2d6.el9.x86_64 3/34 2024-09-05T13:09:19.614 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.3.0-4650.gac9df2d6.el9.x86_64 4/34 2024-09-05T13:09:19.614 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.3.0-4650.gac9df2d6.el9.x86_64 5/34 2024-09-05T13:09:19.614 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.3.0-4650.gac9df2d6.el9.x86_6 6/34 2024-09-05T13:09:19.614 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.3.0-4650.gac9df2d6.el9.x86_64 7/34 2024-09-05T13:09:19.615 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4650.gac9df2d6.el9. 8/34 2024-09-05T13:09:19.615 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.3.0-4650.gac9df2d6.el9.x8 9/34 2024-09-05T13:09:19.615 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.3.0-4650.gac9df2d6.el9.x86_64 10/34 2024-09-05T13:09:19.615 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.3.0-4650.gac9df2d6.el9.x86_64 11/34 2024-09-05T13:09:19.615 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.3.0-4650.gac9df2d6.el9.x86_64 12/34 2024-09-05T13:09:19.615 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.3.0-4650.gac9df2d6.el9.x86_64 13/34 2024-09-05T13:09:19.615 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-05T13:09:19.616 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-05T13:09:19.616 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-05T13:09:19.616 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-05T13:09:19.616 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-05T13:09:19.617 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-05T13:09:19.617 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-05T13:09:19.617 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-05T13:09:19.617 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-05T13:09:19.617 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-09-05T13:09:19.617 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-05T13:09:19.618 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-05T13:09:19.618 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-05T13:09:19.618 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-05T13:09:19.618 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-05T13:09:19.618 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-05T13:09:19.618 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-05T13:09:19.619 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.3.0-4650.gac9df2d6.el9.x86_64 31/34 2024-09-05T13:09:19.619 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-05T13:09:19.619 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.3.0-4650.gac9df2d6.el9.x86_64 33/34 2024-09-05T13:09:20.791 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-05T13:09:20.791 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:20.791 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-09-05T13:09:20.791 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.791 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.791 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:09:20.791 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T13:09:20.791 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T13:09:20.792 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.793 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:20.794 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T13:09:20.794 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T13:09:20.794 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T13:09:20.794 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T13:09:20.794 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T13:09:20.794 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:20.794 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:09:21.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-test 2024-09-05T13:09:21.700 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:28 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:09:21.838 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 50 M 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 161 k 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:09:21.839 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:21.840 INFO:teuthology.orchestra.run.smithi086.stdout:Install 4 Packages 2024-09-05T13:09:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:21.840 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 51 M 2024-09-05T13:09:21.840 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 216 M 2024-09-05T13:09:21.841 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:09:22.310 INFO:teuthology.orchestra.run.smithi086.stdout:(1/4): libcephsqlite-19.3.0-4650.gac9df2d6.el9. 728 kB/s | 161 kB 00:00 2024-09-05T13:09:22.919 INFO:teuthology.orchestra.run.smithi086.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 366 kB/s | 304 kB 00:00 2024-09-05T13:09:22.944 INFO:teuthology.orchestra.run.smithi086.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 102 kB/s | 64 kB 00:00 2024-09-05T13:09:24.103 INFO:teuthology.orchestra.run.smithi086.stdout:(4/4): ceph-test-19.3.0-4650.gac9df2d6.el9.x86_ 25 MB/s | 50 MB 00:02 2024-09-05T13:09:24.105 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:09:24.105 INFO:teuthology.orchestra.run.smithi086.stdout:Total 22 MB/s | 51 MB 00:02 2024-09-05T13:09:24.133 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:09:24.154 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:09:24.154 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:09:24.534 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:09:24.534 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:09:25.067 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:09:25.156 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-05T13:09:25.193 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-05T13:09:25.221 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.3.0-4650.gac9df2d6.el9.x86_64 3/4 2024-09-05T13:09:29.940 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4650.gac9df2d6.el9.x86_64 3/4 2024-09-05T13:09:30.012 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.3.0-4650.gac9df2d6.el9.x86_64 4/4 2024-09-05T13:09:32.336 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-test-2:19.3.0-4650.gac9df2d6.el9.x86_64 4/4 2024-09-05T13:09:32.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/4 2024-09-05T13:09:32.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.3.0-4650.gac9df2d6.el9.x86_64 2/4 2024-09-05T13:09:32.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:32.619 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:09:32.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph 2024-09-05T13:09:33.443 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:40 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:09:33.584 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:09:33.587 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:33.587 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:09:33.587 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:33.587 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:09:33.587 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 6.6 k 2024-09-05T13:09:33.587 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T13:09:33.587 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 2.4 M 2024-09-05T13:09:33.587 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 1.5 M 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 291 k 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 4.8 M 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 17 M 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T13:09:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T13:09:33.589 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:33.590 INFO:teuthology.orchestra.run.smithi086.stdout:Install 37 Packages 2024-09-05T13:09:33.591 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:33.591 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 31 M 2024-09-05T13:09:33.591 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 109 M 2024-09-05T13:09:33.591 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:09:34.289 INFO:teuthology.orchestra.run.smithi086.stdout:(1/37): ceph-19.3.0-4650.gac9df2d6.el9.x86_64.r 51 kB/s | 6.6 kB 00:00 2024-09-05T13:09:34.473 INFO:teuthology.orchestra.run.smithi086.stdout:(2/37): ceph-mgr-19.3.0-4650.gac9df2d6.el9.x86_ 4.9 MB/s | 1.5 MB 00:00 2024-09-05T13:09:34.573 INFO:teuthology.orchestra.run.smithi086.stdout:(3/37): ceph-mds-19.3.0-4650.gac9df2d6.el9.x86_ 5.7 MB/s | 2.4 MB 00:00 2024-09-05T13:09:34.690 INFO:teuthology.orchestra.run.smithi086.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4650.gac9d 2.4 MB/s | 291 kB 00:00 2024-09-05T13:09:34.915 INFO:teuthology.orchestra.run.smithi086.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-05T13:09:35.107 INFO:teuthology.orchestra.run.smithi086.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 6.4 MB/s | 1.2 MB 00:00 2024-09-05T13:09:35.165 INFO:teuthology.orchestra.run.smithi086.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-05T13:09:35.233 INFO:teuthology.orchestra.run.smithi086.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-05T13:09:35.408 INFO:teuthology.orchestra.run.smithi086.stdout:(9/37): ceph-osd-19.3.0-4650.gac9df2d6.el9.x86_ 18 MB/s | 17 MB 00:00 2024-09-05T13:09:35.433 INFO:teuthology.orchestra.run.smithi086.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 624 kB/s | 125 kB 00:00 2024-09-05T13:09:35.500 INFO:teuthology.orchestra.run.smithi086.stdout:(11/37): ceph-mon-19.3.0-4650.gac9df2d6.el9.x86 4.0 MB/s | 4.8 MB 00:01 2024-09-05T13:09:35.692 INFO:teuthology.orchestra.run.smithi086.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 757 kB/s | 215 kB 00:00 2024-09-05T13:09:36.159 INFO:teuthology.orchestra.run.smithi086.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 260 kB/s | 188 kB 00:00 2024-09-05T13:09:36.257 INFO:teuthology.orchestra.run.smithi086.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 61 kB/s | 35 kB 00:00 2024-09-05T13:09:36.283 INFO:teuthology.orchestra.run.smithi086.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-09-05T13:09:36.849 INFO:teuthology.orchestra.run.smithi086.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2024-09-05T13:09:36.875 INFO:teuthology.orchestra.run.smithi086.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-05T13:09:36.900 INFO:teuthology.orchestra.run.smithi086.stdout:(18/37): python3-packaging-20.9-5.el9.noarch.rp 104 kB/s | 77 kB 00:00 2024-09-05T13:09:36.942 INFO:teuthology.orchestra.run.smithi086.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-05T13:09:36.967 INFO:teuthology.orchestra.run.smithi086.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 7.3 MB/s | 172 kB 00:00 2024-09-05T13:09:36.992 INFO:teuthology.orchestra.run.smithi086.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 158 kB/s | 22 kB 00:00 2024-09-05T13:09:37.017 INFO:teuthology.orchestra.run.smithi086.stdout:(22/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 371 kB/s | 43 kB 00:00 2024-09-05T13:09:37.051 INFO:teuthology.orchestra.run.smithi086.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 322 kB/s | 11 kB 00:00 2024-09-05T13:09:37.076 INFO:teuthology.orchestra.run.smithi086.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 998 kB/s | 23 kB 00:00 2024-09-05T13:09:37.102 INFO:teuthology.orchestra.run.smithi086.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 211 kB/s | 18 kB 00:00 2024-09-05T13:09:37.127 INFO:teuthology.orchestra.run.smithi086.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-05T13:09:37.152 INFO:teuthology.orchestra.run.smithi086.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-05T13:09:37.177 INFO:teuthology.orchestra.run.smithi086.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 924 kB/s | 46 kB 00:00 2024-09-05T13:09:37.203 INFO:teuthology.orchestra.run.smithi086.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-05T13:09:37.228 INFO:teuthology.orchestra.run.smithi086.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-05T13:09:37.253 INFO:teuthology.orchestra.run.smithi086.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-05T13:09:37.287 INFO:teuthology.orchestra.run.smithi086.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.1 MB/s | 36 kB 00:00 2024-09-05T13:09:37.312 INFO:teuthology.orchestra.run.smithi086.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-09-05T13:09:37.338 INFO:teuthology.orchestra.run.smithi086.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-05T13:09:37.371 INFO:teuthology.orchestra.run.smithi086.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 604 kB/s | 20 kB 00:00 2024-09-05T13:09:37.396 INFO:teuthology.orchestra.run.smithi086.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-05T13:09:37.421 INFO:teuthology.orchestra.run.smithi086.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-05T13:09:37.427 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:09:37.427 INFO:teuthology.orchestra.run.smithi086.stdout:Total 8.0 MB/s | 31 MB 00:03 2024-09-05T13:09:37.576 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:09:37.616 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:09:37.616 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:09:37.915 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:09:37.916 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:09:39.285 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:09:39.341 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-05T13:09:39.373 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-05T13:09:39.402 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-05T13:09:39.471 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-05T13:09:39.508 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-05T13:09:39.541 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-05T13:09:39.632 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-05T13:09:39.712 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-05T13:09:39.753 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-05T13:09:39.787 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-05T13:09:39.817 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-05T13:09:39.852 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-05T13:09:39.885 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-05T13:09:39.926 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-05T13:09:39.960 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-05T13:09:39.995 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-05T13:09:40.035 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-05T13:09:40.066 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-05T13:09:40.140 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-05T13:09:40.277 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-05T13:09:40.358 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-05T13:09:40.411 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-05T13:09:40.458 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-05T13:09:40.507 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-05T13:09:40.570 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-05T13:09:40.756 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-05T13:09:40.803 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-05T13:09:40.873 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-05T13:09:40.970 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-05T13:09:41.010 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-05T13:09:41.137 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-05T13:09:41.308 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4650.gac9df2d6.el9. 32/37 2024-09-05T13:09:41.346 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.3.0-4650.gac9df2d6.el9.x86_64 33/37 2024-09-05T13:09:41.372 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4650.gac9df2d6.el9.x86_64 33/37 2024-09-05T13:09:41.372 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T13:09:41.372 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T13:09:41.372 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T13:09:41.372 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T13:09:41.372 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:42.830 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.3.0-4650.gac9df2d6.el9.x86_64 34/37 2024-09-05T13:09:42.857 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.3.0-4650.gac9df2d6.el9.x86_64 34/37 2024-09-05T13:09:42.857 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T13:09:42.857 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T13:09:42.857 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T13:09:42.857 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T13:09:42.857 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:43.306 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.3.0-4650.gac9df2d6.el9.x86_64 35/37 2024-09-05T13:09:43.329 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.3.0-4650.gac9df2d6.el9.x86_64 35/37 2024-09-05T13:09:43.330 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T13:09:43.330 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T13:09:43.330 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T13:09:43.330 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T13:09:43.330 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:43.556 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.3.0-4650.gac9df2d6.el9.x86_64 36/37 2024-09-05T13:09:43.580 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.3.0-4650.gac9df2d6.el9.x86_64 36/37 2024-09-05T13:09:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T13:09:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T13:09:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T13:09:43.580 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T13:09:43.580 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:43.643 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.3.0-4650.gac9df2d6.el9.x86_64 37/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-2:19.3.0-4650.gac9df2d6.el9.x86_64 37/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.3.0-4650.gac9df2d6.el9.x86_64 2/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.3.0-4650.gac9df2d6.el9.x86_64 3/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.3.0-4650.gac9df2d6.el9.x86_64 4/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.3.0-4650.gac9df2d6.el9.x86_64 5/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4650.gac9df2d6.el9. 6/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-05T13:09:45.504 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-05T13:09:45.505 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-05T13:09:45.506 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-05T13:09:45.506 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-05T13:09:45.506 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-05T13:09:45.506 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-05T13:09:45.507 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-05T13:09:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-05T13:09:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:46.591 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:09:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:46.591 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T13:09:46.592 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T13:09:46.593 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:46.594 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:09:46.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-base 2024-09-05T13:09:47.516 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:54 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:09:47.605 INFO:teuthology.orchestra.run.smithi086.stdout:Package ceph-base-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:09:47.648 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:09:47.648 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:09:47.648 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:09:47.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install cephadm 2024-09-05T13:09:48.266 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:55 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:09:48.398 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 783 k 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-05T13:09:48.400 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:48.401 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 783 k 2024-09-05T13:09:48.401 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 789 k 2024-09-05T13:09:48.401 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:09:48.679 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm-19.3.0-4650.gac9df2d6.el9.noarch.rpm 2.7 MB/s | 783 kB 00:00 2024-09-05T13:09:48.679 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:09:48.680 INFO:teuthology.orchestra.run.smithi086.stdout:Total 2.7 MB/s | 783 kB 00:00 2024-09-05T13:09:48.680 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:09:48.688 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:09:48.688 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:09:48.691 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:09:48.691 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:09:48.760 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:09:49.192 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 1/1 2024-09-05T13:09:49.281 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 1/1 2024-09-05T13:09:49.611 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 1/1 2024-09-05T13:09:49.847 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 1/1 2024-09-05T13:09:49.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:09:49.847 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:09:49.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:09:50.040 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-immutable-object-cache 2024-09-05T13:09:50.602 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:57 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:09:50.736 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:09:50.736 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repo Size 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 145 k 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 145 k 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 432 k 2024-09-05T13:09:50.737 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:09:50.919 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-immutable-object-cache-19.3.0-4650.gac9df2 800 kB/s | 145 kB 00:00 2024-09-05T13:09:50.920 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:09:50.920 INFO:teuthology.orchestra.run.smithi086.stdout:Total 792 kB/s | 145 kB 00:00 2024-09-05T13:09:50.920 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:09:50.928 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:09:50.928 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:09:50.984 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:09:50.984 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:09:51.128 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:09:51.212 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4650.gac9df2d6. 1/1 2024-09-05T13:09:51.236 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4650.gac9df2d6. 1/1 2024-09-05T13:09:51.237 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T13:09:51.237 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T13:09:51.237 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:52.133 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4650.gac9df2d6. 1/1 2024-09-05T13:09:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:52.134 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:09:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:09:52.134 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:52.134 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:09:52.346 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr 2024-09-05T13:09:52.904 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:59 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:09:52.996 INFO:teuthology.orchestra.run.smithi086.stdout:Package ceph-mgr-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:09:53.038 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:09:53.039 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:09:53.039 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:09:53.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr-dashboard 2024-09-05T13:09:53.654 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:00 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:09:53.784 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:09:53.785 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:53.785 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:09:53.785 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:53.785 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:09:53.785 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 14 M 2024-09-05T13:09:53.785 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 34 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 16 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:53.786 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:09:53.787 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:09:53.787 INFO:teuthology.orchestra.run.smithi086.stdout:Install 12 Packages 2024-09-05T13:09:53.787 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:09:53.787 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 19 M 2024-09-05T13:09:53.787 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 158 M 2024-09-05T13:09:53.787 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:09:54.289 INFO:teuthology.orchestra.run.smithi086.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4650.gac 240 kB/s | 34 kB 00:00 2024-09-05T13:09:54.314 INFO:teuthology.orchestra.run.smithi086.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4650.gac9 98 kB/s | 16 kB 00:00 2024-09-05T13:09:54.439 INFO:teuthology.orchestra.run.smithi086.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-05T13:09:54.523 INFO:teuthology.orchestra.run.smithi086.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-09-05T13:09:54.573 INFO:teuthology.orchestra.run.smithi086.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-05T13:09:54.598 INFO:teuthology.orchestra.run.smithi086.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-05T13:09:54.665 INFO:teuthology.orchestra.run.smithi086.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-05T13:09:54.690 INFO:teuthology.orchestra.run.smithi086.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-05T13:09:54.716 INFO:teuthology.orchestra.run.smithi086.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-05T13:09:54.749 INFO:teuthology.orchestra.run.smithi086.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.7 MB/s | 863 kB 00:00 2024-09-05T13:09:54.774 INFO:teuthology.orchestra.run.smithi086.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-09-05T13:09:55.358 INFO:teuthology.orchestra.run.smithi086.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4650.gac9df2 12 MB/s | 14 MB 00:01 2024-09-05T13:09:55.362 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:09:55.362 INFO:teuthology.orchestra.run.smithi086.stdout:Total 12 MB/s | 19 MB 00:01 2024-09-05T13:09:55.414 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:09:55.434 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:09:55.434 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:09:55.658 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:09:55.660 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:09:56.265 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:09:56.341 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-05T13:09:56.385 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-05T13:09:56.431 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-05T13:09:56.470 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-05T13:09:56.546 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-05T13:09:56.679 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-05T13:09:56.765 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-05T13:09:56.808 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-05T13:09:56.837 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-05T13:09:56.869 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4650.gac9df2d6.el9 10/12 2024-09-05T13:10:00.044 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4650.gac9df2d6.el 11/12 2024-09-05T13:10:00.098 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4650.gac9df2d6.el9.noa 12/12 2024-09-05T13:10:02.072 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4650.gac9df2d6.el9.noa 12/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4650.gac9df2d6.el 1/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4650.gac9df2d6.el9.noa 2/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4650.gac9df2d6.el9 3/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-05T13:10:02.073 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-05T13:10:02.364 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-05T13:10:02.364 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:02.364 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:02.364 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:02.365 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:02.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-05T13:10:03.174 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:10 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:03.307 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 7.4 M 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T13:10:03.309 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout:Install 12 Packages 2024-09-05T13:10:03.310 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:03.311 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 38 M 2024-09-05T13:10:03.311 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 212 M 2024-09-05T13:10:03.311 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:03.605 INFO:teuthology.orchestra.run.smithi086.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-05T13:10:03.828 INFO:teuthology.orchestra.run.smithi086.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 801 kB 00:00 2024-09-05T13:10:03.872 INFO:teuthology.orchestra.run.smithi086.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 122 kB/s | 33 kB 00:00 2024-09-05T13:10:03.939 INFO:teuthology.orchestra.run.smithi086.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 272 kB/s | 18 kB 00:00 2024-09-05T13:10:04.014 INFO:teuthology.orchestra.run.smithi086.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 514 kB/s | 38 kB 00:00 2024-09-05T13:10:04.339 INFO:teuthology.orchestra.run.smithi086.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-46 8.2 MB/s | 7.4 MB 00:00 2024-09-05T13:10:04.731 INFO:teuthology.orchestra.run.smithi086.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 625 kB/s | 245 kB 00:00 2024-09-05T13:10:05.132 INFO:teuthology.orchestra.run.smithi086.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.0 MB/s | 3.0 MB 00:01 2024-09-05T13:10:05.256 INFO:teuthology.orchestra.run.smithi086.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-05T13:10:05.790 INFO:teuthology.orchestra.run.smithi086.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.8 MB/s | 4.9 MB 00:01 2024-09-05T13:10:06.557 INFO:teuthology.orchestra.run.smithi086.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.8 MB/s | 5.1 MB 00:01 2024-09-05T13:10:09.041 INFO:teuthology.orchestra.run.smithi086.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.3 MB/s | 16 MB 00:03 2024-09-05T13:10:09.042 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:09.042 INFO:teuthology.orchestra.run.smithi086.stdout:Total 6.7 MB/s | 38 MB 00:05 2024-09-05T13:10:09.406 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:09.432 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:09.432 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:09.845 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:09.846 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:10.600 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:10.678 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-05T13:10:10.713 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-05T13:10:10.814 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-05T13:10:10.856 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-05T13:10:11.042 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-05T13:10:11.071 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-05T13:10:11.173 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-05T13:10:11.820 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-05T13:10:11.942 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-05T13:10:13.173 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-05T13:10:14.208 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-05T13:10:14.244 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4650.gac9df 12/12 2024-09-05T13:10:16.742 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4650.gac9df 12/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4650.gac9df 1/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-05T13:10:16.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-05T13:10:17.087 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-05T13:10:17.087 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:17.087 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:17.087 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:10:17.087 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T13:10:17.087 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T13:10:17.087 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T13:10:17.087 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:17.088 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:17.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr-rook 2024-09-05T13:10:18.025 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:25 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:18.157 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:18.158 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:10:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:18.158 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 49 k 2024-09-05T13:10:18.158 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T13:10:18.158 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T13:10:18.158 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:18.159 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:Install 13 Packages 2024-09-05T13:10:18.160 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 2.2 M 2024-09-05T13:10:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 26 M 2024-09-05T13:10:18.160 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:18.386 INFO:teuthology.orchestra.run.smithi086.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 235 kB/s | 26 kB 00:00 2024-09-05T13:10:18.403 INFO:teuthology.orchestra.run.smithi086.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 152 kB/s | 19 kB 00:00 2024-09-05T13:10:18.461 INFO:teuthology.orchestra.run.smithi086.stdout:(3/13): ceph-mgr-rook-19.3.0-4650.gac9df2d6.el9 267 kB/s | 49 kB 00:00 2024-09-05T13:10:18.487 INFO:teuthology.orchestra.run.smithi086.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-05T13:10:18.528 INFO:teuthology.orchestra.run.smithi086.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-05T13:10:18.571 INFO:teuthology.orchestra.run.smithi086.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 639 kB/s | 54 kB 00:00 2024-09-05T13:10:18.604 INFO:teuthology.orchestra.run.smithi086.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 425 kB/s | 32 kB 00:00 2024-09-05T13:10:18.629 INFO:teuthology.orchestra.run.smithi086.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 249 kB/s | 14 kB 00:00 2024-09-05T13:10:18.663 INFO:teuthology.orchestra.run.smithi086.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.8 MB/s | 220 kB 00:00 2024-09-05T13:10:18.688 INFO:teuthology.orchestra.run.smithi086.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-05T13:10:18.714 INFO:teuthology.orchestra.run.smithi086.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-05T13:10:18.764 INFO:teuthology.orchestra.run.smithi086.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-05T13:10:18.789 INFO:teuthology.orchestra.run.smithi086.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 895 kB/s | 90 kB 00:00 2024-09-05T13:10:18.791 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:18.792 INFO:teuthology.orchestra.run.smithi086.stdout:Total 3.5 MB/s | 2.2 MB 00:00 2024-09-05T13:10:18.854 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:18.870 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:18.870 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:18.958 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:18.959 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:19.462 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:19.627 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-05T13:10:19.674 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-05T13:10:19.729 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-05T13:10:19.766 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-05T13:10:19.807 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-05T13:10:19.917 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-05T13:10:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-05T13:10:20.076 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-05T13:10:20.794 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-05T13:10:20.849 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-05T13:10:20.881 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-05T13:10:20.929 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-05T13:10:20.951 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.3.0-4650.gac9df2d6.el9.noarch 13/13 2024-09-05T13:10:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4650.gac9df2d6.el9.noarch 13/13 2024-09-05T13:10:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4650.gac9df2d6.el9.noarch 1/13 2024-09-05T13:10:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-05T13:10:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-05T13:10:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-05T13:10:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:22.005 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T13:10:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:22.007 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:22.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-mgr-cephadm 2024-09-05T13:10:22.815 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:29 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:22.948 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 154 k 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T13:10:22.949 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T13:10:22.950 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T13:10:22.950 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T13:10:22.950 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:22.950 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:22.950 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:22.950 INFO:teuthology.orchestra.run.smithi086.stdout:Install 6 Packages 2024-09-05T13:10:22.950 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:22.951 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 7.0 M 2024-09-05T13:10:22.951 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 33 M 2024-09-05T13:10:22.951 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:23.306 INFO:teuthology.orchestra.run.smithi086.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4650.gac9df2d6.e 674 kB/s | 154 kB 00:00 2024-09-05T13:10:23.357 INFO:teuthology.orchestra.run.smithi086.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-05T13:10:23.382 INFO:teuthology.orchestra.run.smithi086.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-05T13:10:23.407 INFO:teuthology.orchestra.run.smithi086.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-05T13:10:23.666 INFO:teuthology.orchestra.run.smithi086.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 422 kB/s | 248 kB 00:00 2024-09-05T13:10:26.132 INFO:teuthology.orchestra.run.smithi086.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.0 MB/s | 6.0 MB 00:03 2024-09-05T13:10:26.134 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:26.135 INFO:teuthology.orchestra.run.smithi086.stdout:Total 2.2 MB/s | 7.0 MB 00:03 2024-09-05T13:10:26.207 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:26.219 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:26.219 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:26.318 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:26.319 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:26.622 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:26.779 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-05T13:10:26.825 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-05T13:10:27.347 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-05T13:10:27.428 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-05T13:10:27.504 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-05T13:10:27.547 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 6/6 2024-09-05T13:10:28.265 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 6/6 2024-09-05T13:10:28.266 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 1/6 2024-09-05T13:10:28.266 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-05T13:10:28.266 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-05T13:10:28.266 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-05T13:10:28.266 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-05T13:10:28.516 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-05T13:10:28.516 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:28.516 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:28.517 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:10:28.517 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T13:10:28.517 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T13:10:28.517 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T13:10:28.517 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T13:10:28.517 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T13:10:28.517 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:28.517 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:28.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-fuse 2024-09-05T13:10:29.312 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:36 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:29.443 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 900 k 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:29.444 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:29.445 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:29.445 INFO:teuthology.orchestra.run.smithi086.stdout:Install 2 Packages 2024-09-05T13:10:29.445 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:29.445 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 979 k 2024-09-05T13:10:29.446 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 2.9 M 2024-09-05T13:10:29.446 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:29.583 INFO:teuthology.orchestra.run.smithi086.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 869 kB/s | 80 kB 00:00 2024-09-05T13:10:29.809 INFO:teuthology.orchestra.run.smithi086.stdout:(2/2): ceph-fuse-19.3.0-4650.gac9df2d6.el9.x86_ 2.8 MB/s | 900 kB 00:00 2024-09-05T13:10:29.809 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:29.809 INFO:teuthology.orchestra.run.smithi086.stdout:Total 2.6 MB/s | 979 kB 00:00 2024-09-05T13:10:29.821 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:29.842 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:29.843 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:29.922 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:29.922 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:30.107 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:30.252 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-05T13:10:30.323 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.3.0-4650.gac9df2d6.el9.x86_64 2/2 2024-09-05T13:10:30.999 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4650.gac9df2d6.el9.x86_64 2/2 2024-09-05T13:10:31.000 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/2 2024-09-05T13:10:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-05T13:10:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:31.242 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:31.242 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.3.0-4650.gac9df2d6.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-05T13:10:31.243 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:31.243 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:31.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-volume 2024-09-05T13:10:32.082 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:39 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:32.220 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.3.0-4650.gac9df2d6.el9 ceph-noarch 285 k 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-05T13:10:32.221 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:32.222 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 285 k 2024-09-05T13:10:32.222 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 1.3 M 2024-09-05T13:10:32.222 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:32.462 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-volume-19.3.0-4650.gac9df2d6.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-05T13:10:32.463 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:32.463 INFO:teuthology.orchestra.run.smithi086.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-05T13:10:32.463 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:32.471 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:32.471 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:32.495 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:32.495 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:32.719 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:32.789 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.3.0-4650.gac9df2d6.el9.noarch 1/1 2024-09-05T13:10:32.805 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.3.0-4650.gac9df2d6.el9.noarch 1/1 2024-09-05T13:10:32.805 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T13:10:32.805 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T13:10:32.805 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.3.0-4650.gac9df2d6.el9.noarch 1/1 2024-09-05T13:10:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:33.761 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.3.0-4650.gac9df2d6.el9.noarch 2024-09-05T13:10:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:33.761 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:34.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install librados-devel 2024-09-05T13:10:34.602 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:41 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:34.737 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:34.737 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:34.737 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repo Size 2024-09-05T13:10:34.737 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 127 k 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 127 k 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 456 k 2024-09-05T13:10:34.738 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:librados-devel-19.3.0-4650.gac9df2d6.el9.x86_64 576 kB/s | 127 kB 00:00 2024-09-05T13:10:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:34.958 INFO:teuthology.orchestra.run.smithi086.stdout:Total 572 kB/s | 127 kB 00:00 2024-09-05T13:10:34.959 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:34.964 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:34.964 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:35.028 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:35.028 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:35.174 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:35.270 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:35.677 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados-devel-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:35.911 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:10:35.911 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:35.911 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:36.166 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install libcephfs2 2024-09-05T13:10:36.736 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:43 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:36.828 INFO:teuthology.orchestra.run.smithi086.stdout:Package libcephfs2-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:10:36.873 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:36.873 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:10:36.874 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:36.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install libcephfs-devel 2024-09-05T13:10:37.489 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:44 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:37.621 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repo Size 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 32 k 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 32 k 2024-09-05T13:10:37.622 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 142 k 2024-09-05T13:10:37.623 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:37.777 INFO:teuthology.orchestra.run.smithi086.stdout:libcephfs-devel-19.3.0-4650.gac9df2d6.el9.x86_6 206 kB/s | 32 kB 00:00 2024-09-05T13:10:37.777 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:37.777 INFO:teuthology.orchestra.run.smithi086.stdout:Total 204 kB/s | 32 kB 00:00 2024-09-05T13:10:37.778 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:37.781 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:37.782 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:37.785 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:37.785 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:37.865 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:37.973 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:38.349 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:38.566 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:10:38.566 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:38.567 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:38.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install librados2 2024-09-05T13:10:39.295 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:46 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:39.383 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:10:39.426 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:39.427 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:10:39.427 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:39.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install librbd1 2024-09-05T13:10:40.047 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:47 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:40.136 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:10:40.179 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:40.180 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:10:40.180 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:40.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install python3-rados 2024-09-05T13:10:40.806 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:47 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:40.895 INFO:teuthology.orchestra.run.smithi086.stdout:Package python3-rados-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:10:40.938 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:40.939 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:10:40.939 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:41.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install python3-rgw 2024-09-05T13:10:41.560 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:48 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:41.649 INFO:teuthology.orchestra.run.smithi086.stdout:Package python3-rgw-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:10:41.691 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:41.692 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:10:41.692 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:41.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install python3-cephfs 2024-09-05T13:10:42.320 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:49 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:42.407 INFO:teuthology.orchestra.run.smithi086.stdout:Package python3-cephfs-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:10:42.450 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:42.451 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:10:42.451 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:42.526 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install python3-rbd 2024-09-05T13:10:43.079 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:50 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:43.169 INFO:teuthology.orchestra.run.smithi086.stdout:Package python3-rbd-2:19.3.0-4650.gac9df2d6.el9.x86_64 is already installed. 2024-09-05T13:10:43.212 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:43.213 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-09-05T13:10:43.213 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:43.293 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install rbd-fuse 2024-09-05T13:10:43.848 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:50 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:43.983 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:43.984 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:43.984 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-05T13:10:43.984 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:43.984 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:43.984 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 85 k 2024-09-05T13:10:43.984 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:43.985 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:43.985 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:43.985 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-05T13:10:43.985 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:43.985 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 85 k 2024-09-05T13:10:43.985 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 231 k 2024-09-05T13:10:43.985 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:44.180 INFO:teuthology.orchestra.run.smithi086.stdout:rbd-fuse-19.3.0-4650.gac9df2d6.el9.x86_64.rpm 434 kB/s | 85 kB 00:00 2024-09-05T13:10:44.181 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:44.181 INFO:teuthology.orchestra.run.smithi086.stdout:Total 431 kB/s | 85 kB 00:00 2024-09-05T13:10:44.181 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:44.188 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:44.188 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:44.253 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:44.253 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:44.394 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:44.476 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:44.830 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:45.081 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:45.081 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:45.081 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:45.081 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:10:45.081 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:45.081 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:45.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install rbd-mirror 2024-09-05T13:10:45.914 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:52 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:46.053 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:46.053 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:46.053 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repo Size 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 3.2 M 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 3.2 M 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 14 M 2024-09-05T13:10:46.054 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:46.500 INFO:teuthology.orchestra.run.smithi086.stdout:rbd-mirror-19.3.0-4650.gac9df2d6.el9.x86_64.rpm 7.1 MB/s | 3.2 MB 00:00 2024-09-05T13:10:46.500 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:46.501 INFO:teuthology.orchestra.run.smithi086.stdout:Total 7.1 MB/s | 3.2 MB 00:00 2024-09-05T13:10:46.501 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:46.510 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:46.510 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:46.592 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:46.592 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:47.041 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:47.109 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T13:10:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T13:10:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T13:10:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T13:10:47.135 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:48.250 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:10:48.250 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:48.251 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:48.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install rbd-nbd 2024-09-05T13:10:49.086 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:01:56 ago on Thu 05 Sep 2024 01:08:53 PM UTC. 2024-09-05T13:10:49.221 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.3.0-4650.gac9df2d6.el9 ceph 171 k 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-09-05T13:10:49.222 INFO:teuthology.orchestra.run.smithi086.stdout:Install 1 Package 2024-09-05T13:10:49.223 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:49.223 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 171 k 2024-09-05T13:10:49.223 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 490 k 2024-09-05T13:10:49.223 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-09-05T13:10:49.427 INFO:teuthology.orchestra.run.smithi086.stdout:rbd-nbd-19.3.0-4650.gac9df2d6.el9.x86_64.rpm 834 kB/s | 171 kB 00:00 2024-09-05T13:10:49.428 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-09-05T13:10:49.428 INFO:teuthology.orchestra.run.smithi086.stdout:Total 829 kB/s | 171 kB 00:00 2024-09-05T13:10:49.428 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-09-05T13:10:49.436 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-09-05T13:10:49.436 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-09-05T13:10:49.501 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-09-05T13:10:49.501 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-09-05T13:10:49.652 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-09-05T13:10:49.748 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:50.132 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:50.376 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.3.0-4650.gac9df2d6.el9.x86_64 1/1 2024-09-05T13:10:50.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:50.377 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-09-05T13:10:50.377 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.3.0-4650.gac9df2d6.el9.x86_64 2024-09-05T13:10:50.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:50.377 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-09-05T13:10:50.648 DEBUG:teuthology.parallel:result is None 2024-09-05T13:10:50.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03 2024-09-05T13:10:50.809 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T13:10:50.830 INFO:teuthology.orchestra.run.smithi086.stdout:19.3.0-4650.gac9df2d6.el9 2024-09-05T13:10:50.831 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4650.gac9df2d6.el9 2024-09-05T13:10:50.831 INFO:teuthology.task.install:The correct ceph version 19.3.0-4650.gac9df2d6 is installed. 2024-09-05T13:10:50.833 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-05T13:10:50.833 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:50.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T13:10:50.897 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-05T13:10:50.897 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:50.897 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T13:10:50.962 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T13:10:51.026 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-05T13:10:51.027 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:51.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T13:10:51.091 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T13:10:51.155 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-05T13:10:51.155 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:51.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T13:10:51.220 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T13:10:51.285 INFO:teuthology.run_tasks:Running task ceph... 2024-09-05T13:10:51.372 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-05T13:10:51.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2024-09-05T13:10:51.396 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-05T13:10:51.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-05T13:10:51.461 INFO:tasks.ceph:Creating extra log directories... 2024-09-05T13:10:51.462 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-05T13:10:51.533 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-05T13:10:51.533 INFO:tasks.ceph:config {'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-09-05T13:10:51.534 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478', 'branch': 'wip-bharath4-testing-2024-09-02-0957', 'description': 'rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-bitmap} rados supported-random-distro$/{centos_latest}}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7890478', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath4-testing-2024-09-02-0957'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'mon_bind_msgr2': False, 'sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath4-testing-2024-09-02-0957', 'sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 5171, 'sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath4-testing-2024-09-02-0957', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03', 'targets': {'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNFW74Bw72hrhPkH0GuhcO/JNalMqdIhZBuVS0J+H97uL9ZLUloQvoNDA/+pcgGNqxXvUKMaMFqmPIAiHl/iV9c='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'mon_bind_msgr2': False, 'sha1': 'ac9df2d66a4c8e4e8634a3fe39168f4f4683bc03', 'cluster': 'ceph'}}, {'rep_lost_unfound_delete': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-05_12:54:40', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-05T13:10:51.534 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-05T13:10:51.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2024-09-05T13:10:51.646 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:51.646 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T13:10:51.702 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T13:10:51.702 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 13:10:47.395839240 +0000 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 13:08:10.058747627 +0000 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 13:08:10.058747627 +0000 2024-09-05T13:10:51.757 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-05 13:08:10.058747627 +0000 2024-09-05T13:10:51.758 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T13:10:51.820 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-05T13:10:51.820 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-05T13:10:51.820 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000222708 s, 2.3 MB/s 2024-09-05T13:10:51.821 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T13:10:51.878 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 13:10:47.395839240 +0000 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 13:08:10.391740342 +0000 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 13:08:10.391740342 +0000 2024-09-05T13:10:51.933 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-05 13:08:10.391740342 +0000 2024-09-05T13:10:51.934 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T13:10:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-05T13:10:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-05T13:10:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000218752 s, 2.3 MB/s 2024-09-05T13:10:51.997 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T13:10:52.053 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 13:10:47.396839218 +0000 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 13:08:10.721733123 +0000 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 13:08:10.721733123 +0000 2024-09-05T13:10:52.108 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-05 13:08:10.721733123 +0000 2024-09-05T13:10:52.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T13:10:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-05T13:10:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-05T13:10:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000164761 s, 3.1 MB/s 2024-09-05T13:10:52.172 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T13:10:52.228 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-09-05 13:10:47.396839218 +0000 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-09-05 13:08:11.052725882 +0000 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-09-05 13:08:11.052725882 +0000 2024-09-05T13:10:52.283 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-09-05 13:08:11.052725882 +0000 2024-09-05T13:10:52.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T13:10:52.345 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-09-05T13:10:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-09-05T13:10:52.346 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000221512 s, 2.3 MB/s 2024-09-05T13:10:52.347 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T13:10:52.404 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-05T13:10:52.404 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-05T13:10:52.404 INFO:tasks.ceph:Generating config... 2024-09-05T13:10:52.406 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-05T13:10:52.406 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-05T13:10:52.406 INFO:tasks.ceph:[global] ms type = async 2024-09-05T13:10:52.406 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-05T13:10:52.406 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-05T13:10:52.406 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-05T13:10:52.406 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-05T13:10:52.406 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-05T13:10:52.406 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] bluestore write v2 = False 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-05T13:10:52.407 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-05T13:10:52.408 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-05T13:10:52.408 INFO:tasks.ceph:Setting up mon.a... 2024-09-05T13:10:52.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-05T13:10:52.494 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.keyring 2024-09-05T13:10:52.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-05T13:10:52.539 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-05T13:10:52.603 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.86:6789]'), ('mon.b', '[v1:172.21.15.86:6790]'), ('mon.c', '[v1:172.21.15.86:6791]')] 2024-09-05T13:10:52.604 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.86:6789],[v1:172.21.15.86:6790],[v1:172.21.15.86:6791]', 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-05T13:10:52.605 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:52.605 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-05T13:10:52.659 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.86:6789]' --addv b '[v1:172.21.15.86:6790]' --addv c '[v1:172.21.15.86:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-05T13:10:52.745 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-05T13:10:52.745 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: generated fsid 0162b639-bdbb-4bd7-a784-515f78b22316 2024-09-05T13:10:52.745 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-09-05T13:10:52.745 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 0 2024-09-05T13:10:52.745 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 0162b639-bdbb-4bd7-a784-515f78b22316 2024-09-05T13:10:52.745 INFO:teuthology.orchestra.run.smithi086.stdout:last_changed 2024-09-05T13:10:52.744886+0000 2024-09-05T13:10:52.745 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-09-05T13:10:52.744886+0000 2024-09-05T13:10:52.746 INFO:teuthology.orchestra.run.smithi086.stdout:min_mon_release 17 (quincy) 2024-09-05T13:10:52.746 INFO:teuthology.orchestra.run.smithi086.stdout:election_strategy: 3 2024-09-05T13:10:52.746 INFO:teuthology.orchestra.run.smithi086.stdout:0: v1:172.21.15.86:6789/0 mon.a 2024-09-05T13:10:52.746 INFO:teuthology.orchestra.run.smithi086.stdout:1: v1:172.21.15.86:6790/0 mon.b 2024-09-05T13:10:52.746 INFO:teuthology.orchestra.run.smithi086.stdout:2: v1:172.21.15.86:6791/0 mon.c 2024-09-05T13:10:52.746 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-05T13:10:52.747 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-05T13:10:52.762 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0162b639-bdbb-4bd7-a784-515f78b22316... 2024-09-05T13:10:52.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-05T13:10:52.846 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-09-05T13:10:52.846 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-09-05T13:10:52.846 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-05T13:10:52.846 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-09-05T13:10:52.846 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.846 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-09-05T13:10:52.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-09-05T13:10:52.848 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-09-05T13:10:52.849 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "[v1:172.21.15.86:6789],[v1:172.21.15.86:6790],[v1:172.21.15.86:6791]" 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: mon election default strategy = 3 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: ms bind msgr2 = False 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: ms type = async 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 0162b639-bdbb-4bd7-a784-515f78b22316 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.850 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-09-05T13:10:52.851 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: bdev async discard = True 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: bdev enable discard = True 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = bitmap 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-09-05T13:10:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore write v2 = False 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore zero block detection = True 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify cached snaps = True 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify missing on start = True 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-09-05T13:10:52.853 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock override recovery settings = True 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: osd op queue = debug_random 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: osd op queue cut off = debug_random 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.854 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-05T13:10:52.855 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: mon scrub interval = 300 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:10:52.856 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-09-05T13:10:52.857 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-09-05T13:10:52.857 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-09-05T13:10:52.857 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-09-05T13:10:52.857 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-05T13:10:52.857 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-05T13:10:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-09-05T13:10:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-09-05T13:10:52.857 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-09-05T13:10:52.858 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-05T13:10:52.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-05T13:10:52.952 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-05T13:10:52.952 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:52.952 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-05T13:10:52.967 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:52.967 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-05T13:10:53.022 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:10:53.022 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:53.022 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-05T13:10:53.022 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-05T13:10:53.096 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:10:53.096 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-05T13:10:53.151 INFO:tasks.ceph:Setting up mon nodes... 2024-09-05T13:10:53.151 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-05T13:10:53.151 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-05T13:10:53.241 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-05T13:10:53.244 INFO:tasks.ceph:Setting up mds nodes... 2024-09-05T13:10:53.244 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-05T13:10:53.244 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-05T13:10:53.244 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-05T13:10:53.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-05T13:10:53.268 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-05T13:10:53.268 INFO:tasks.ceph:role: osd.0 2024-09-05T13:10:53.268 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:10:53.269 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-05T13:10:53.340 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-05T13:10:53.340 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-05T13:10:53.340 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-05T13:10:53.341 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-05T13:10:53.341 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-05T13:10:53.341 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-09-05T13:10:53.341 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-05T13:10:53.341 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-05T13:10:53.341 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-05T13:10:53.341 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-05T13:10:53.842 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-09-05T13:10:53.844 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-09-05T13:10:53.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-05T13:10:53.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-05T13:10:54.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-05T13:10:54.077 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-05T13:10:54.077 INFO:tasks.ceph:role: osd.1 2024-09-05T13:10:54.077 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:10:54.078 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-05T13:10:54.151 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-05T13:10:54.611 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-09-05T13:10:54.613 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-09-05T13:10:54.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-05T13:10:54.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-05T13:10:54.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-05T13:10:54.794 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-05T13:10:54.794 INFO:tasks.ceph:role: osd.2 2024-09-05T13:10:54.795 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:10:54.795 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-05T13:10:54.866 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-05T13:10:54.866 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-05T13:10:54.867 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-05T13:10:54.867 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-05T13:10:54.867 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-05T13:10:54.867 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-09-05T13:10:54.867 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-05T13:10:54.867 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-05T13:10:54.867 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-05T13:10:54.867 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-05T13:10:55.352 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-09-05T13:10:55.354 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-09-05T13:10:55.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-05T13:10:55.503 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-05T13:10:55.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-05T13:10:55.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:10:55.626+0000 7fae22ed8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-05T13:10:55.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:10:55.629+0000 7fae22ed8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-05T13:10:55.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:10:55.629+0000 7fae22ed8640 -1 bdev(0x5578fe13d800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-05T13:10:55.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:10:55.630+0000 7fae22ed8640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-05T13:10:59.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-05T13:10:59.075 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-05T13:10:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:10:59.162+0000 7fee9f21b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-05T13:10:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:10:59.162+0000 7fee9f21b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-05T13:10:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:10:59.162+0000 7fee9f21b640 -1 bdev(0x55f679a1b800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-05T13:10:59.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:10:59.162+0000 7fee9f21b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-05T13:11:02.569 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-05T13:11:02.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-05T13:11:02.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:11:02.688+0000 7fd77ae9c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-05T13:11:02.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:11:02.689+0000 7fd77ae9c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-05T13:11:02.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:11:02.689+0000 7fd77ae9c640 -1 bdev(0x563a499e3800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-05T13:11:02.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-09-05T13:11:02.689+0000 7fd77ae9c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-05T13:11:05.615 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-05T13:11:05.642 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-05T13:11:05.642 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:11:05.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-05T13:11:05.709 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:11:05.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-05T13:11:05.775 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:11:05.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-05T13:11:05.842 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:11:05.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-05T13:11:05.907 INFO:tasks.ceph:Adding keys to all mons... 2024-09-05T13:11:05.908 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-05T13:11:05.971 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2024-09-05T13:11:05.971 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDdrdlmvTFjDhAAaE6G0xcH8KT1NrFjIhrwcA== 2024-09-05T13:11:05.972 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2024-09-05T13:11:05.972 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDfrdlmlE1fJRAAYN4ytWriu9auj/VjKw9Qhg== 2024-09-05T13:11:05.972 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2024-09-05T13:11:05.972 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDjrdlmJ8+8CRAAxaL5+U1uR3Lbc4TRw7d0Ww== 2024-09-05T13:11:05.972 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2024-09-05T13:11:05.972 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDmrdlm5wscKRAAsDMRI3HchVieEF1UNyYV7A== 2024-09-05T13:11:05.973 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-05T13:11:06.063 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-05T13:11:06.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-05T13:11:06.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-05T13:11:06.219 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-05T13:11:06.219 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-05T13:11:06.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-05T13:11:06.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-05T13:11:06.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-05T13:11:06.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-05T13:11:06.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-05T13:11:06.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-05T13:11:06.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-05T13:11:06.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-05T13:11:06.735 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-05T13:11:06.790 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-05T13:11:06.790 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-05T13:11:06.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-05T13:11:06.832 INFO:tasks.ceph.mon.a:Started 2024-09-05T13:11:06.832 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-05T13:11:06.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-05T13:11:06.835 INFO:tasks.ceph.mon.b:Started 2024-09-05T13:11:06.835 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-05T13:11:06.835 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-05T13:11:06.837 INFO:tasks.ceph.mon.c:Started 2024-09-05T13:11:06.837 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph config set mgr mgr_pool false --force 2024-09-05T13:11:13.243 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-05T13:11:13.244 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-05T13:11:13.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-05T13:11:13.246 INFO:tasks.ceph.mgr.x:Started 2024-09-05T13:11:13.246 DEBUG:tasks.ceph:set 0 configs 2024-09-05T13:11:13.247 INFO:tasks.ceph:Setting crush tunables to default 2024-09-05T13:11:13.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd crush tunables default 2024-09-05T13:11:13.473 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-09-05T13:11:13.481 INFO:tasks.ceph:check_enable_crimson: False 2024-09-05T13:11:13.482 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-05T13:11:13.482 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:11:13.482 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-05T13:11:13.506 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:11:13.506 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-05T13:11:13.570 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-09-05T13:11:13.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-05T13:11:13.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new df7aef1c-c387-49d7-b4b7-627cbd8ab67e 0 2024-09-05T13:11:13.881 INFO:teuthology.orchestra.run.smithi086.stdout:0 2024-09-05T13:11:13.889 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 623d69d0-4fe7-431e-aa67-d7239d6aef62 1 2024-09-05T13:11:14.084 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-09-05T13:11:14.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new b7c8e230-5ce4-4513-b07a-64cfea1afc1e 2 2024-09-05T13:11:14.273 INFO:teuthology.orchestra.run.smithi086.stdout:2 2024-09-05T13:11:14.284 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-05T13:11:14.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-05T13:11:14.290 INFO:tasks.ceph.osd.0:Started 2024-09-05T13:11:14.290 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-05T13:11:14.290 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-05T13:11:14.292 INFO:tasks.ceph.osd.1:Started 2024-09-05T13:11:14.292 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-05T13:11:14.293 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-05T13:11:14.295 INFO:tasks.ceph.osd.2:Started 2024-09-05T13:11:14.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T13:11:14.475 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:14.476 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":5,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:11:14.269193+0000","last_up_change":"0.000000","last_in_change":"2024-09-05T13:11:14.269193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:11:14.487 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-05T13:11:14.488 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-05T13:11:14.789 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-05T13:11:14.947 INFO:teuthology.misc.health.smithi086.stdout: 2024-09-05T13:11:14.948 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":5,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:11:14.269193+0000","last_up_change":"0.000000","last_in_change":"2024-09-05T13:11:14.269193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:11:14.961 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-05T13:11:15.285 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-09-05T13:11:15.284+0000 7ff0ee7cc640 -1 Falling back to public interface 2024-09-05T13:11:15.794 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:11:15.793+0000 7f2063e1e640 -1 Falling back to public interface 2024-09-05T13:11:16.246 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-09-05T13:11:16.245+0000 7fcef8cd4640 -1 Falling back to public interface 2024-09-05T13:11:18.503 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-05T13:11:18.501+0000 7f81db475640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-05T13:11:18.503 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-09-05T13:11:18.501+0000 7f81db475640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-05T13:11:19.395 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-09-05T13:11:19.393+0000 7ff0ee7cc640 -1 osd.1 0 log_to_monitors true 2024-09-05T13:11:19.626 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:11:19.625+0000 7f2063e1e640 -1 osd.0 0 log_to_monitors true 2024-09-05T13:11:20.042 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-09-05T13:11:20.041+0000 7fcef8cd4640 -1 osd.2 0 log_to_monitors true 2024-09-05T13:11:20.507 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-09-05T13:11:20.505+0000 7ff0ea75b640 -1 osd.1 0 waiting for initial osdmap 2024-09-05T13:11:20.510 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-09-05T13:11:20.509+0000 7ff0e6582640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T13:11:21.263 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-05T13:11:21.509 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-09-05T13:11:21.507+0000 7fcef4c63640 -1 osd.2 0 waiting for initial osdmap 2024-09-05T13:11:21.523 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-09-05T13:11:21.522+0000 7fcef0a8a640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T13:11:21.558 INFO:teuthology.misc.health.smithi086.stdout: 2024-09-05T13:11:21.558 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":8,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:11:21.501617+0000","last_up_change":"2024-09-05T13:11:21.501617+0000","last_in_change":"2024-09-05T13:11:14.269193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2999410110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2999410110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2999410110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2999410110}]},"public_addr":"172.21.15.86:6800/2999410110","cluster_addr":"172.21.15.86:6801/2999410110","heartbeat_back_addr":"172.21.15.86:6803/2999410110","heartbeat_front_addr":"172.21.15.86:6802/2999410110","state":["exists","up"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:11:21.570 DEBUG:teuthology.misc:1 of 3 OSDs are up 2024-09-05T13:11:22.209 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:11:22.208+0000 7f2060072640 -1 osd.0 0 waiting for initial osdmap 2024-09-05T13:11:22.212 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:11:22.211+0000 7f205767d640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T13:11:27.872 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-05T13:11:28.156 INFO:teuthology.misc.health.smithi086.stdout: 2024-09-05T13:11:28.156 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":10,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:11:23.518185+0000","last_up_change":"2024-09-05T13:11:22.512943+0000","last_in_change":"2024-09-05T13:11:14.269193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6804","nonce":2426100778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6805","nonce":2426100778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6807","nonce":2426100778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6806","nonce":2426100778}]},"public_addr":"172.21.15.86:6804/2426100778","cluster_addr":"172.21.15.86:6805/2426100778","heartbeat_back_addr":"172.21.15.86:6807/2426100778","heartbeat_front_addr":"172.21.15.86:6806/2426100778","state":["exists","up"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2999410110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2999410110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2999410110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2999410110}]},"public_addr":"172.21.15.86:6800/2999410110","cluster_addr":"172.21.15.86:6801/2999410110","heartbeat_back_addr":"172.21.15.86:6803/2999410110","heartbeat_front_addr":"172.21.15.86:6802/2999410110","state":["exists","up"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6808","nonce":184901110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6809","nonce":184901110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6811","nonce":184901110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6810","nonce":184901110}]},"public_addr":"172.21.15.86:6808/184901110","cluster_addr":"172.21.15.86:6809/184901110","heartbeat_back_addr":"172.21.15.86:6811/184901110","heartbeat_front_addr":"172.21.15.86:6810/184901110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.643488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.440515+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:21.020419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:11:28.167 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-05T13:11:28.167 INFO:tasks.ceph:Creating RBD pool 2024-09-05T13:11:28.167 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-05T13:11:28.536 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'rbd' created 2024-09-05T13:11:28.549 DEBUG:teuthology.orchestra.run.smithi086:> rbd --cluster ceph pool init rbd 2024-09-05T13:11:31.577 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-05T13:11:31.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-05T13:11:31.578 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-05T13:11:31.867 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:31.880 INFO:teuthology.orchestra.run.smithi086.stdout:[{"version":3,"timestamp":"2024-09-05T13:11:22.209701+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"11833.737503"}]}] 2024-09-05T13:11:31.880 INFO:tasks.ceph_manager:config epoch is 3 2024-09-05T13:11:31.880 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-05T13:11:31.880 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-05T13:11:31.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-05T13:11:32.208 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:32.219 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6812","nonce":3651025766}]},"active_addr":"172.21.15.86:6812/3651025766","active_change":"2024-09-05T13:11:17.490153+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3186027338}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":94885936}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1938363600}]}]} 2024-09-05T13:11:32.223 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-05T13:11:32.223 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-05T13:11:32.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T13:11:32.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:32.505 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":14,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:11:31.550844+0000","last_up_change":"2024-09-05T13:11:22.512943+0000","last_in_change":"2024-09-05T13:11:14.269193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-05T13:11:28.452062+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6804","nonce":2426100778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6805","nonce":2426100778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6807","nonce":2426100778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6806","nonce":2426100778}]},"public_addr":"172.21.15.86:6804/2426100778","cluster_addr":"172.21.15.86:6805/2426100778","heartbeat_back_addr":"172.21.15.86:6807/2426100778","heartbeat_front_addr":"172.21.15.86:6806/2426100778","state":["exists","up"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2999410110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2999410110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2999410110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2999410110}]},"public_addr":"172.21.15.86:6800/2999410110","cluster_addr":"172.21.15.86:6801/2999410110","heartbeat_back_addr":"172.21.15.86:6803/2999410110","heartbeat_front_addr":"172.21.15.86:6802/2999410110","state":["exists","up"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6808","nonce":184901110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6809","nonce":184901110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6811","nonce":184901110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6810","nonce":184901110}]},"public_addr":"172.21.15.86:6808/184901110","cluster_addr":"172.21.15.86:6809/184901110","heartbeat_back_addr":"172.21.15.86:6811/184901110","heartbeat_front_addr":"172.21.15.86:6810/184901110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.643488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.440515+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:21.020419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:11:32.518 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-05T13:11:32.518 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T13:11:32.800 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:32.800 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":14,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:11:31.550844+0000","last_up_change":"2024-09-05T13:11:22.512943+0000","last_in_change":"2024-09-05T13:11:14.269193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-05T13:11:28.452062+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6804","nonce":2426100778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6805","nonce":2426100778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6807","nonce":2426100778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6806","nonce":2426100778}]},"public_addr":"172.21.15.86:6804/2426100778","cluster_addr":"172.21.15.86:6805/2426100778","heartbeat_back_addr":"172.21.15.86:6807/2426100778","heartbeat_front_addr":"172.21.15.86:6806/2426100778","state":["exists","up"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2999410110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2999410110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2999410110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2999410110}]},"public_addr":"172.21.15.86:6800/2999410110","cluster_addr":"172.21.15.86:6801/2999410110","heartbeat_back_addr":"172.21.15.86:6803/2999410110","heartbeat_front_addr":"172.21.15.86:6802/2999410110","state":["exists","up"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6808","nonce":184901110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6809","nonce":184901110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6811","nonce":184901110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6810","nonce":184901110}]},"public_addr":"172.21.15.86:6808/184901110","cluster_addr":"172.21.15.86:6809/184901110","heartbeat_back_addr":"172.21.15.86:6811/184901110","heartbeat_front_addr":"172.21.15.86:6810/184901110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.643488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.440515+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:21.020419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:11:32.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-05T13:11:32.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-05T13:11:32.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-05T13:11:32.950 INFO:teuthology.orchestra.run.smithi086.stdout:38654705669 2024-09-05T13:11:32.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:11:32.954 INFO:teuthology.orchestra.run.smithi086.stdout:34359738373 2024-09-05T13:11:32.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:11:32.955 INFO:teuthology.orchestra.run.smithi086.stdout:38654705668 2024-09-05T13:11:32.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:11:33.277 INFO:teuthology.orchestra.run.smithi086.stdout:38654705668 2024-09-05T13:11:33.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-09-05T13:11:33.292 INFO:teuthology.orchestra.run.smithi086.stdout:38654705667 2024-09-05T13:11:33.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2024-09-05T13:11:33.316 INFO:teuthology.orchestra.run.smithi086.stdout:34359738371 2024-09-05T13:11:33.328 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.1 2024-09-05T13:11:34.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:11:34.305 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:11:34.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:11:34.625 INFO:teuthology.orchestra.run.smithi086.stdout:38654705669 2024-09-05T13:11:34.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-09-05T13:11:34.638 DEBUG:teuthology.parallel:result is None 2024-09-05T13:11:34.640 INFO:teuthology.orchestra.run.smithi086.stdout:38654705668 2024-09-05T13:11:34.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.2 2024-09-05T13:11:34.653 DEBUG:teuthology.parallel:result is None 2024-09-05T13:11:34.658 INFO:teuthology.orchestra.run.smithi086.stdout:34359738373 2024-09-05T13:11:34.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.1 2024-09-05T13:11:34.671 DEBUG:teuthology.parallel:result is None 2024-09-05T13:11:34.672 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-05T13:11:34.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:11:34.990 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:34.990 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:11:35.002 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-05T13:11:33.499221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81448,"kb_used_data":360,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034072,"statfs":{"total":289910292480,"available":289826889728,"internally_reserved":0,"allocated":368640,"data_stored":84351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562296+0000","last_change":"2024-09-05T13:11:31.562296+0000","last_active":"2024-09-05T13:11:31.562296+0000","last_peered":"2024-09-05T13:11:31.562296+0000","last_clean":"2024-09-05T13:11:31.562296+0000","last_became_active":"2024-09-05T13:11:29.558450+0000","last_became_peered":"2024-09-05T13:11:29.558450+0000","last_unstale":"2024-09-05T13:11:31.562296+0000","last_undegraded":"2024-09-05T13:11:31.562296+0000","last_fullsized":"2024-09-05T13:11:31.562296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:11:44.748402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.561623+0000","last_change":"2024-09-05T13:11:31.561623+0000","last_active":"2024-09-05T13:11:31.561623+0000","last_peered":"2024-09-05T13:11:31.561623+0000","last_clean":"2024-09-05T13:11:31.561623+0000","last_became_active":"2024-09-05T13:11:29.550478+0000","last_became_peered":"2024-09-05T13:11:29.550478+0000","last_unstale":"2024-09-05T13:11:31.561623+0000","last_undegraded":"2024-09-05T13:11:31.561623+0000","last_fullsized":"2024-09-05T13:11:31.561623+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:20:35.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.964392+0000","last_change":"2024-09-05T13:11:31.964392+0000","last_active":"2024-09-05T13:11:31.964392+0000","last_peered":"2024-09-05T13:11:31.964392+0000","last_clean":"2024-09-05T13:11:31.964392+0000","last_became_active":"2024-09-05T13:11:29.558123+0000","last_became_peered":"2024-09-05T13:11:29.558123+0000","last_unstale":"2024-09-05T13:11:31.964392+0000","last_undegraded":"2024-09-05T13:11:31.964392+0000","last_fullsized":"2024-09-05T13:11:31.964392+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:46:33.266386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039787699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562474+0000","last_change":"2024-09-05T13:11:31.562474+0000","last_active":"2024-09-05T13:11:31.562474+0000","last_peered":"2024-09-05T13:11:31.562474+0000","last_clean":"2024-09-05T13:11:31.562474+0000","last_became_active":"2024-09-05T13:11:29.550485+0000","last_became_peered":"2024-09-05T13:11:29.550485+0000","last_unstale":"2024-09-05T13:11:31.562474+0000","last_undegraded":"2024-09-05T13:11:31.562474+0000","last_fullsized":"2024-09-05T13:11:31.562474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:42:09.992680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562553+0000","last_change":"2024-09-05T13:11:31.562553+0000","last_active":"2024-09-05T13:11:31.562553+0000","last_peered":"2024-09-05T13:11:31.562553+0000","last_clean":"2024-09-05T13:11:31.562553+0000","last_became_active":"2024-09-05T13:11:29.558411+0000","last_became_peered":"2024-09-05T13:11:29.558411+0000","last_unstale":"2024-09-05T13:11:31.562553+0000","last_undegraded":"2024-09-05T13:11:31.562553+0000","last_fullsized":"2024-09-05T13:11:31.562553+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:20:16.795153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.564148+0000","last_change":"2024-09-05T13:11:31.561639+0000","last_active":"2024-09-05T13:11:31.564148+0000","last_peered":"2024-09-05T13:11:31.564148+0000","last_clean":"2024-09-05T13:11:31.564148+0000","last_became_active":"2024-09-05T13:11:29.551153+0000","last_became_peered":"2024-09-05T13:11:29.551153+0000","last_unstale":"2024-09-05T13:11:31.564148+0000","last_undegraded":"2024-09-05T13:11:31.564148+0000","last_fullsized":"2024-09-05T13:11:31.564148+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:15:21.585273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.964393+0000","last_change":"2024-09-05T13:11:31.964393+0000","last_active":"2024-09-05T13:11:31.964393+0000","last_peered":"2024-09-05T13:11:31.964393+0000","last_clean":"2024-09-05T13:11:31.964393+0000","last_became_active":"2024-09-05T13:11:29.558157+0000","last_became_peered":"2024-09-05T13:11:29.558157+0000","last_unstale":"2024-09-05T13:11:31.964393+0000","last_undegraded":"2024-09-05T13:11:31.964393+0000","last_fullsized":"2024-09-05T13:11:31.964393+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:30:09.684394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400894,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.561573+0000","last_change":"2024-09-05T13:11:31.561573+0000","last_active":"2024-09-05T13:11:31.561573+0000","last_peered":"2024-09-05T13:11:31.561573+0000","last_clean":"2024-09-05T13:11:31.561573+0000","last_became_active":"2024-09-05T13:11:29.547775+0000","last_became_peered":"2024-09-05T13:11:29.547775+0000","last_unstale":"2024-09-05T13:11:31.561573+0000","last_undegraded":"2024-09-05T13:11:31.561573+0000","last_fullsized":"2024-09-05T13:11:31.561573+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:37:50.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":114688,"data_stored":26267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:11:35.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:11:35.280 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:35.281 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:11:35.292 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-05T13:11:33.499221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81448,"kb_used_data":360,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034072,"statfs":{"total":289910292480,"available":289826889728,"internally_reserved":0,"allocated":368640,"data_stored":84351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562296+0000","last_change":"2024-09-05T13:11:31.562296+0000","last_active":"2024-09-05T13:11:31.562296+0000","last_peered":"2024-09-05T13:11:31.562296+0000","last_clean":"2024-09-05T13:11:31.562296+0000","last_became_active":"2024-09-05T13:11:29.558450+0000","last_became_peered":"2024-09-05T13:11:29.558450+0000","last_unstale":"2024-09-05T13:11:31.562296+0000","last_undegraded":"2024-09-05T13:11:31.562296+0000","last_fullsized":"2024-09-05T13:11:31.562296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:11:44.748402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.561623+0000","last_change":"2024-09-05T13:11:31.561623+0000","last_active":"2024-09-05T13:11:31.561623+0000","last_peered":"2024-09-05T13:11:31.561623+0000","last_clean":"2024-09-05T13:11:31.561623+0000","last_became_active":"2024-09-05T13:11:29.550478+0000","last_became_peered":"2024-09-05T13:11:29.550478+0000","last_unstale":"2024-09-05T13:11:31.561623+0000","last_undegraded":"2024-09-05T13:11:31.561623+0000","last_fullsized":"2024-09-05T13:11:31.561623+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:20:35.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.964392+0000","last_change":"2024-09-05T13:11:31.964392+0000","last_active":"2024-09-05T13:11:31.964392+0000","last_peered":"2024-09-05T13:11:31.964392+0000","last_clean":"2024-09-05T13:11:31.964392+0000","last_became_active":"2024-09-05T13:11:29.558123+0000","last_became_peered":"2024-09-05T13:11:29.558123+0000","last_unstale":"2024-09-05T13:11:31.964392+0000","last_undegraded":"2024-09-05T13:11:31.964392+0000","last_fullsized":"2024-09-05T13:11:31.964392+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:46:33.266386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039787699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562474+0000","last_change":"2024-09-05T13:11:31.562474+0000","last_active":"2024-09-05T13:11:31.562474+0000","last_peered":"2024-09-05T13:11:31.562474+0000","last_clean":"2024-09-05T13:11:31.562474+0000","last_became_active":"2024-09-05T13:11:29.550485+0000","last_became_peered":"2024-09-05T13:11:29.550485+0000","last_unstale":"2024-09-05T13:11:31.562474+0000","last_undegraded":"2024-09-05T13:11:31.562474+0000","last_fullsized":"2024-09-05T13:11:31.562474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:42:09.992680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562553+0000","last_change":"2024-09-05T13:11:31.562553+0000","last_active":"2024-09-05T13:11:31.562553+0000","last_peered":"2024-09-05T13:11:31.562553+0000","last_clean":"2024-09-05T13:11:31.562553+0000","last_became_active":"2024-09-05T13:11:29.558411+0000","last_became_peered":"2024-09-05T13:11:29.558411+0000","last_unstale":"2024-09-05T13:11:31.562553+0000","last_undegraded":"2024-09-05T13:11:31.562553+0000","last_fullsized":"2024-09-05T13:11:31.562553+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:20:16.795153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.564148+0000","last_change":"2024-09-05T13:11:31.561639+0000","last_active":"2024-09-05T13:11:31.564148+0000","last_peered":"2024-09-05T13:11:31.564148+0000","last_clean":"2024-09-05T13:11:31.564148+0000","last_became_active":"2024-09-05T13:11:29.551153+0000","last_became_peered":"2024-09-05T13:11:29.551153+0000","last_unstale":"2024-09-05T13:11:31.564148+0000","last_undegraded":"2024-09-05T13:11:31.564148+0000","last_fullsized":"2024-09-05T13:11:31.564148+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:15:21.585273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.964393+0000","last_change":"2024-09-05T13:11:31.964393+0000","last_active":"2024-09-05T13:11:31.964393+0000","last_peered":"2024-09-05T13:11:31.964393+0000","last_clean":"2024-09-05T13:11:31.964393+0000","last_became_active":"2024-09-05T13:11:29.558157+0000","last_became_peered":"2024-09-05T13:11:29.558157+0000","last_unstale":"2024-09-05T13:11:31.964393+0000","last_undegraded":"2024-09-05T13:11:31.964393+0000","last_fullsized":"2024-09-05T13:11:31.964393+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:30:09.684394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400894,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.561573+0000","last_change":"2024-09-05T13:11:31.561573+0000","last_active":"2024-09-05T13:11:31.561573+0000","last_peered":"2024-09-05T13:11:31.561573+0000","last_clean":"2024-09-05T13:11:31.561573+0000","last_became_active":"2024-09-05T13:11:29.547775+0000","last_became_peered":"2024-09-05T13:11:29.547775+0000","last_unstale":"2024-09-05T13:11:31.561573+0000","last_undegraded":"2024-09-05T13:11:31.561573+0000","last_fullsized":"2024-09-05T13:11:31.561573+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:37:50.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":114688,"data_stored":26267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:11:35.293 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-05T13:11:35.293 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-05T13:11:35.293 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-05T13:11:35.293 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-05T13:11:35.613 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:35.613 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-05T13:11:35.625 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-05T13:11:35.625 INFO:teuthology.run_tasks:Running task rep_lost_unfound_delete... 2024-09-05T13:11:35.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T13:11:35.920 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:35.921 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":14,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:11:31.550844+0000","last_up_change":"2024-09-05T13:11:22.512943+0000","last_in_change":"2024-09-05T13:11:14.269193+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-05T13:11:28.452062+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6804","nonce":2426100778}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6805","nonce":2426100778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6807","nonce":2426100778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6806","nonce":2426100778}]},"public_addr":"172.21.15.86:6804/2426100778","cluster_addr":"172.21.15.86:6805/2426100778","heartbeat_back_addr":"172.21.15.86:6807/2426100778","heartbeat_front_addr":"172.21.15.86:6806/2426100778","state":["exists","up"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2999410110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2999410110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2999410110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2999410110}]},"public_addr":"172.21.15.86:6800/2999410110","cluster_addr":"172.21.15.86:6801/2999410110","heartbeat_back_addr":"172.21.15.86:6803/2999410110","heartbeat_front_addr":"172.21.15.86:6802/2999410110","state":["exists","up"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6808","nonce":184901110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6809","nonce":184901110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6811","nonce":184901110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6810","nonce":184901110}]},"public_addr":"172.21.15.86:6808/184901110","cluster_addr":"172.21.15.86:6809/184901110","heartbeat_back_addr":"172.21.15.86:6811/184901110","heartbeat_front_addr":"172.21.15.86:6810/184901110","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.643488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.440515+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:21.020419+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:11:35.933 INFO:tasks.rep_lost_unfound_delete.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-05T13:11:28.452062+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 14, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-05T13:11:35.933 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-05T13:11:36.215 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 8 2024-09-05T13:11:36.228 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-05T13:11:36.510 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 14 2024-09-05T13:11:36.510 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 0162b639-bdbb-4bd7-a784-515f78b22316 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-09-05T13:11:13.133363+0000 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-09-05T13:11:31.550844+0000 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:crush_version 5 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:full_ratio 0.95 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:backfillfull_ratio 0.9 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:nearfull_ratio 0.85 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:require_min_compat_client luminous 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:min_compat_client jewel 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:require_osd_release squid 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:stretch_mode_enabled false 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-05T13:11:36.511 INFO:teuthology.orchestra.run.smithi086.stdout: removed_snaps_queue [2~1] 2024-09-05T13:11:36.512 INFO:teuthology.orchestra.run.smithi086.stdout:max_osd 3 2024-09-05T13:11:36.512 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.86:6804/2426100778 v1:172.21.15.86:6805/2426100778 exists,up df7aef1c-c387-49d7-b4b7-627cbd8ab67e 2024-09-05T13:11:36.512 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.86:6800/2999410110 v1:172.21.15.86:6801/2999410110 exists,up 623d69d0-4fe7-431e-aa67-d7239d6aef62 2024-09-05T13:11:36.512 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.86:6808/184901110 v1:172.21.15.86:6809/184901110 exists,up b7c8e230-5ce4-4513-b07a-64cfea1afc1e 2024-09-05T13:11:36.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.86:6804/2426100778 v1:172.21.15.86:6805/2426100778 exists,up df7aef1c-c387-49d7-b4b7-627cbd8ab67e', 'osd.1 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.86:6800/2999410110 v1:172.21.15.86:6801/2999410110 exists,up 623d69d0-4fe7-431e-aa67-d7239d6aef62', 'osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.86:6808/184901110 v1:172.21.15.86:6809/184901110 exists,up b7c8e230-5ce4-4513-b07a-64cfea1afc1e'] 2024-09-05T13:11:36.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-05T13:11:36.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-05T13:11:36.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-05T13:11:36.659 INFO:teuthology.orchestra.run.smithi086.stdout:38654705671 2024-09-05T13:11:36.660 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:11:36.668 INFO:teuthology.orchestra.run.smithi086.stdout:34359738374 2024-09-05T13:11:36.669 INFO:teuthology.orchestra.run.smithi086.stdout:38654705670 2024-09-05T13:11:36.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:11:36.670 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:11:36.959 INFO:teuthology.orchestra.run.smithi086.stdout:38654705670 2024-09-05T13:11:36.971 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705671 got 38654705670 for osd.0 2024-09-05T13:11:37.014 INFO:teuthology.orchestra.run.smithi086.stdout:34359738373 2024-09-05T13:11:37.015 INFO:teuthology.orchestra.run.smithi086.stdout:38654705669 2024-09-05T13:11:37.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738374 got 34359738373 for osd.1 2024-09-05T13:11:37.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705670 got 38654705669 for osd.2 2024-09-05T13:11:37.973 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:11:38.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:11:38.030 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:11:38.302 INFO:teuthology.orchestra.run.smithi086.stdout:38654705671 2024-09-05T13:11:38.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705671 got 38654705671 for osd.0 2024-09-05T13:11:38.316 DEBUG:teuthology.parallel:result is None 2024-09-05T13:11:38.332 INFO:teuthology.orchestra.run.smithi086.stdout:38654705670 2024-09-05T13:11:38.344 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705670 got 38654705670 for osd.2 2024-09-05T13:11:38.344 DEBUG:teuthology.parallel:result is None 2024-09-05T13:11:38.348 INFO:teuthology.orchestra.run.smithi086.stdout:34359738374 2024-09-05T13:11:38.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738374 got 34359738374 for osd.1 2024-09-05T13:11:38.361 DEBUG:teuthology.parallel:result is None 2024-09-05T13:11:38.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-05T13:11:38.361 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:11:38.680 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:38.680 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:11:38.691 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-05T13:11:37.499902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81460,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034060,"statfs":{"total":289910292480,"available":289826877440,"internally_reserved":0,"allocated":376832,"data_stored":87107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000681"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562296+0000","last_change":"2024-09-05T13:11:31.562296+0000","last_active":"2024-09-05T13:11:31.562296+0000","last_peered":"2024-09-05T13:11:31.562296+0000","last_clean":"2024-09-05T13:11:31.562296+0000","last_became_active":"2024-09-05T13:11:29.558450+0000","last_became_peered":"2024-09-05T13:11:29.558450+0000","last_unstale":"2024-09-05T13:11:31.562296+0000","last_undegraded":"2024-09-05T13:11:31.562296+0000","last_fullsized":"2024-09-05T13:11:31.562296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:11:44.748402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.561623+0000","last_change":"2024-09-05T13:11:31.561623+0000","last_active":"2024-09-05T13:11:31.561623+0000","last_peered":"2024-09-05T13:11:31.561623+0000","last_clean":"2024-09-05T13:11:31.561623+0000","last_became_active":"2024-09-05T13:11:29.550478+0000","last_became_peered":"2024-09-05T13:11:29.550478+0000","last_unstale":"2024-09-05T13:11:31.561623+0000","last_undegraded":"2024-09-05T13:11:31.561623+0000","last_fullsized":"2024-09-05T13:11:31.561623+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:20:35.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.964392+0000","last_change":"2024-09-05T13:11:31.964392+0000","last_active":"2024-09-05T13:11:31.964392+0000","last_peered":"2024-09-05T13:11:31.964392+0000","last_clean":"2024-09-05T13:11:31.964392+0000","last_became_active":"2024-09-05T13:11:29.558123+0000","last_became_peered":"2024-09-05T13:11:29.558123+0000","last_unstale":"2024-09-05T13:11:31.964392+0000","last_undegraded":"2024-09-05T13:11:31.964392+0000","last_fullsized":"2024-09-05T13:11:31.964392+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:46:33.266386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039787699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562474+0000","last_change":"2024-09-05T13:11:31.562474+0000","last_active":"2024-09-05T13:11:31.562474+0000","last_peered":"2024-09-05T13:11:31.562474+0000","last_clean":"2024-09-05T13:11:31.562474+0000","last_became_active":"2024-09-05T13:11:29.550485+0000","last_became_peered":"2024-09-05T13:11:29.550485+0000","last_unstale":"2024-09-05T13:11:31.562474+0000","last_undegraded":"2024-09-05T13:11:31.562474+0000","last_fullsized":"2024-09-05T13:11:31.562474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:42:09.992680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562553+0000","last_change":"2024-09-05T13:11:31.562553+0000","last_active":"2024-09-05T13:11:31.562553+0000","last_peered":"2024-09-05T13:11:31.562553+0000","last_clean":"2024-09-05T13:11:31.562553+0000","last_became_active":"2024-09-05T13:11:29.558411+0000","last_became_peered":"2024-09-05T13:11:29.558411+0000","last_unstale":"2024-09-05T13:11:31.562553+0000","last_undegraded":"2024-09-05T13:11:31.562553+0000","last_fullsized":"2024-09-05T13:11:31.562553+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:20:16.795153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.564148+0000","last_change":"2024-09-05T13:11:31.561639+0000","last_active":"2024-09-05T13:11:31.564148+0000","last_peered":"2024-09-05T13:11:31.564148+0000","last_clean":"2024-09-05T13:11:31.564148+0000","last_became_active":"2024-09-05T13:11:29.551153+0000","last_became_peered":"2024-09-05T13:11:29.551153+0000","last_unstale":"2024-09-05T13:11:31.564148+0000","last_undegraded":"2024-09-05T13:11:31.564148+0000","last_fullsized":"2024-09-05T13:11:31.564148+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:15:21.585273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.964393+0000","last_change":"2024-09-05T13:11:31.964393+0000","last_active":"2024-09-05T13:11:31.964393+0000","last_peered":"2024-09-05T13:11:31.964393+0000","last_clean":"2024-09-05T13:11:31.964393+0000","last_became_active":"2024-09-05T13:11:29.558157+0000","last_became_peered":"2024-09-05T13:11:29.558157+0000","last_unstale":"2024-09-05T13:11:31.964393+0000","last_undegraded":"2024-09-05T13:11:31.964393+0000","last_fullsized":"2024-09-05T13:11:31.964393+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:30:09.684394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400894,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.561573+0000","last_change":"2024-09-05T13:11:31.561573+0000","last_active":"2024-09-05T13:11:31.561573+0000","last_peered":"2024-09-05T13:11:31.561573+0000","last_clean":"2024-09-05T13:11:31.561573+0000","last_became_active":"2024-09-05T13:11:29.547775+0000","last_became_peered":"2024-09-05T13:11:29.547775+0000","last_unstale":"2024-09-05T13:11:31.561573+0000","last_undegraded":"2024-09-05T13:11:31.561573+0000","last_fullsized":"2024-09-05T13:11:31.561573+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:37:50.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":122880,"data_stored":29023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:11:38.692 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:11:38.972 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:38.972 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:11:38.983 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-05T13:11:37.499902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81460,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034060,"statfs":{"total":289910292480,"available":289826877440,"internally_reserved":0,"allocated":376832,"data_stored":87107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000681"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562296+0000","last_change":"2024-09-05T13:11:31.562296+0000","last_active":"2024-09-05T13:11:31.562296+0000","last_peered":"2024-09-05T13:11:31.562296+0000","last_clean":"2024-09-05T13:11:31.562296+0000","last_became_active":"2024-09-05T13:11:29.558450+0000","last_became_peered":"2024-09-05T13:11:29.558450+0000","last_unstale":"2024-09-05T13:11:31.562296+0000","last_undegraded":"2024-09-05T13:11:31.562296+0000","last_fullsized":"2024-09-05T13:11:31.562296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:11:44.748402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.561623+0000","last_change":"2024-09-05T13:11:31.561623+0000","last_active":"2024-09-05T13:11:31.561623+0000","last_peered":"2024-09-05T13:11:31.561623+0000","last_clean":"2024-09-05T13:11:31.561623+0000","last_became_active":"2024-09-05T13:11:29.550478+0000","last_became_peered":"2024-09-05T13:11:29.550478+0000","last_unstale":"2024-09-05T13:11:31.561623+0000","last_undegraded":"2024-09-05T13:11:31.561623+0000","last_fullsized":"2024-09-05T13:11:31.561623+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:20:35.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.964392+0000","last_change":"2024-09-05T13:11:31.964392+0000","last_active":"2024-09-05T13:11:31.964392+0000","last_peered":"2024-09-05T13:11:31.964392+0000","last_clean":"2024-09-05T13:11:31.964392+0000","last_became_active":"2024-09-05T13:11:29.558123+0000","last_became_peered":"2024-09-05T13:11:29.558123+0000","last_unstale":"2024-09-05T13:11:31.964392+0000","last_undegraded":"2024-09-05T13:11:31.964392+0000","last_fullsized":"2024-09-05T13:11:31.964392+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:46:33.266386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039787699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562474+0000","last_change":"2024-09-05T13:11:31.562474+0000","last_active":"2024-09-05T13:11:31.562474+0000","last_peered":"2024-09-05T13:11:31.562474+0000","last_clean":"2024-09-05T13:11:31.562474+0000","last_became_active":"2024-09-05T13:11:29.550485+0000","last_became_peered":"2024-09-05T13:11:29.550485+0000","last_unstale":"2024-09-05T13:11:31.562474+0000","last_undegraded":"2024-09-05T13:11:31.562474+0000","last_fullsized":"2024-09-05T13:11:31.562474+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:42:09.992680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.562553+0000","last_change":"2024-09-05T13:11:31.562553+0000","last_active":"2024-09-05T13:11:31.562553+0000","last_peered":"2024-09-05T13:11:31.562553+0000","last_clean":"2024-09-05T13:11:31.562553+0000","last_became_active":"2024-09-05T13:11:29.558411+0000","last_became_peered":"2024-09-05T13:11:29.558411+0000","last_unstale":"2024-09-05T13:11:31.562553+0000","last_undegraded":"2024-09-05T13:11:31.562553+0000","last_fullsized":"2024-09-05T13:11:31.562553+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:20:16.795153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.564148+0000","last_change":"2024-09-05T13:11:31.561639+0000","last_active":"2024-09-05T13:11:31.564148+0000","last_peered":"2024-09-05T13:11:31.564148+0000","last_clean":"2024-09-05T13:11:31.564148+0000","last_became_active":"2024-09-05T13:11:29.551153+0000","last_became_peered":"2024-09-05T13:11:29.551153+0000","last_unstale":"2024-09-05T13:11:31.564148+0000","last_undegraded":"2024-09-05T13:11:31.564148+0000","last_fullsized":"2024-09-05T13:11:31.564148+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:15:21.585273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.964393+0000","last_change":"2024-09-05T13:11:31.964393+0000","last_active":"2024-09-05T13:11:31.964393+0000","last_peered":"2024-09-05T13:11:31.964393+0000","last_clean":"2024-09-05T13:11:31.964393+0000","last_became_active":"2024-09-05T13:11:29.558157+0000","last_became_peered":"2024-09-05T13:11:29.558157+0000","last_unstale":"2024-09-05T13:11:31.964393+0000","last_undegraded":"2024-09-05T13:11:31.964393+0000","last_fullsized":"2024-09-05T13:11:31.964393+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:30:09.684394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000400894,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-05T13:11:31.561573+0000","last_change":"2024-09-05T13:11:31.561573+0000","last_active":"2024-09-05T13:11:31.561573+0000","last_peered":"2024-09-05T13:11:31.561573+0000","last_clean":"2024-09-05T13:11:31.561573+0000","last_became_active":"2024-09-05T13:11:29.547775+0000","last_became_peered":"2024-09-05T13:11:29.547775+0000","last_unstale":"2024-09-05T13:11:31.561573+0000","last_undegraded":"2024-09-05T13:11:31.561573+0000","last_fullsized":"2024-09-05T13:11:31.561573+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:37:50.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":122880,"data_stored":29023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:11:38.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-05T13:11:38.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:creating pool_name unfounddel_pool 2024-09-05T13:11:38.984 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfounddel_pool 16 2024-09-05T13:11:39.939 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'unfounddel_pool' created 2024-09-05T13:11:39.954 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfounddel_pool rados --yes-i-really-mean-it || true 2024-09-05T13:11:40.936 INFO:teuthology.orchestra.run.smithi086.stderr:enabled application 'rados' on pool 'unfounddel_pool' 2024-09-05T13:11:41.959 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-05T13:11:41.960 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:11:41.999 INFO:tasks.ceph.osd.2:Stopped 2024-09-05T13:11:41.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-05T13:11:42.345 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:11:42.344+0000 7f9a87bca640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-09-05T13:11:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:marked down osd.2. 2024-09-05T13:11:42.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-05T13:11:43.269 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:11:43.268+0000 7f9a87bca640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-05T13:11:43.963 INFO:teuthology.orchestra.run.smithi086.stderr:marked out osd.2. 2024-09-05T13:11:43.980 INFO:tasks.util.rados:rados -p unfounddel_pool put dummy /etc/fstab 2024-09-05T13:11:43.980 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put dummy /etc/fstab 2024-09-05T13:11:44.030 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-05T13:11:44.030 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-05T13:11:44.179 INFO:teuthology.orchestra.run.smithi086.stdout:34359738377 2024-09-05T13:11:44.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:11:44.180 INFO:teuthology.orchestra.run.smithi086.stdout:38654705674 2024-09-05T13:11:44.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:11:44.509 INFO:teuthology.orchestra.run.smithi086.stdout:38654705672 2024-09-05T13:11:44.510 INFO:teuthology.orchestra.run.smithi086.stdout:34359738376 2024-09-05T13:11:44.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-05T13:11:44.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705674 got 38654705672 for osd.0 2024-09-05T13:11:45.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:11:45.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:11:45.820 INFO:teuthology.orchestra.run.smithi086.stdout:38654705672 2024-09-05T13:11:45.821 INFO:teuthology.orchestra.run.smithi086.stdout:34359738376 2024-09-05T13:11:45.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705674 got 38654705672 for osd.0 2024-09-05T13:11:45.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-05T13:11:46.835 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:11:46.835 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:11:47.134 INFO:teuthology.orchestra.run.smithi086.stdout:38654705674 2024-09-05T13:11:47.135 INFO:teuthology.orchestra.run.smithi086.stdout:34359738377 2024-09-05T13:11:47.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705674 got 38654705674 for osd.0 2024-09-05T13:11:47.148 DEBUG:teuthology.parallel:result is None 2024-09-05T13:11:47.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738377 for osd.1 2024-09-05T13:11:47.148 DEBUG:teuthology.parallel:result is None 2024-09-05T13:11:47.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-05T13:11:47.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:11:47.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:47.425 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:11:47.437 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-05T13:11:45.501183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54392,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689288,"statfs":{"total":193273528320,"available":193217830912,"internally_reserved":0,"allocated":335872,"data_stored":87222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000704"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969424+0000","last_change":"2024-09-05T13:11:40.946256+0000","last_active":"2024-09-05T13:11:43.969424+0000","last_peered":"2024-09-05T13:11:43.969424+0000","last_clean":"2024-09-05T13:11:43.969424+0000","last_became_active":"2024-09-05T13:11:40.945982+0000","last_became_peered":"2024-09-05T13:11:40.945982+0000","last_unstale":"2024-09-05T13:11:43.969424+0000","last_undegraded":"2024-09-05T13:11:43.969424+0000","last_fullsized":"2024-09-05T13:11:43.969424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:28:56.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509608+0000","last_change":"2024-09-05T13:11:42.959184+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509608+0000","last_undegraded":"2024-09-05T13:11:43.509608+0000","last_fullsized":"2024-09-05T13:11:43.509608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.960474+0000","last_change":"2024-09-05T13:11:42.960474+0000","last_active":"2024-09-05T13:11:42.960474+0000","last_peered":"2024-09-05T13:11:42.508089+0000","last_clean":"2024-09-05T13:11:42.508089+0000","last_became_active":"2024-09-05T13:11:40.944583+0000","last_became_peered":"2024-09-05T13:11:40.944583+0000","last_unstale":"2024-09-05T13:11:42.960474+0000","last_undegraded":"2024-09-05T13:11:42.960474+0000","last_fullsized":"2024-09-05T13:11:42.960474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509595+0000","last_change":"2024-09-05T13:11:42.959615+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509595+0000","last_undegraded":"2024-09-05T13:11:43.509595+0000","last_fullsized":"2024-09-05T13:11:43.509595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509587+0000","last_change":"2024-09-05T13:11:42.959048+0000","last_active":"2024-09-05T13:11:42.959048+0000","last_peered":"2024-09-05T13:11:41.951881+0000","last_clean":"2024-09-05T13:11:41.951881+0000","last_became_active":"2024-09-05T13:11:40.942058+0000","last_became_peered":"2024-09-05T13:11:40.942058+0000","last_unstale":"2024-09-05T13:11:43.509587+0000","last_undegraded":"2024-09-05T13:11:43.509587+0000","last_fullsized":"2024-09-05T13:11:43.509587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969835+0000","last_change":"2024-09-05T13:11:40.945249+0000","last_active":"2024-09-05T13:11:43.969835+0000","last_peered":"2024-09-05T13:11:43.969835+0000","last_clean":"2024-09-05T13:11:43.969835+0000","last_became_active":"2024-09-05T13:11:40.945009+0000","last_became_peered":"2024-09-05T13:11:40.945009+0000","last_unstale":"2024-09-05T13:11:43.969835+0000","last_undegraded":"2024-09-05T13:11:43.969835+0000","last_fullsized":"2024-09-05T13:11:43.969835+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:35.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970821+0000","last_change":"2024-09-05T13:11:40.944662+0000","last_active":"2024-09-05T13:11:43.970821+0000","last_peered":"2024-09-05T13:11:43.970821+0000","last_clean":"2024-09-05T13:11:43.970821+0000","last_became_active":"2024-09-05T13:11:40.944287+0000","last_became_peered":"2024-09-05T13:11:40.944287+0000","last_unstale":"2024-09-05T13:11:43.970821+0000","last_undegraded":"2024-09-05T13:11:43.970821+0000","last_fullsized":"2024-09-05T13:11:43.970821+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:15:15.352497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509544+0000","last_change":"2024-09-05T13:11:42.960433+0000","last_active":"2024-09-05T13:11:42.960433+0000","last_peered":"2024-09-05T13:11:41.951561+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:11:40.941932+0000","last_became_peered":"2024-09-05T13:11:40.941932+0000","last_unstale":"2024-09-05T13:11:43.509544+0000","last_undegraded":"2024-09-05T13:11:43.509544+0000","last_fullsized":"2024-09-05T13:11:43.509544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970998+0000","last_change":"2024-09-05T13:11:31.561573+0000","last_active":"2024-09-05T13:11:43.970998+0000","last_peered":"2024-09-05T13:11:43.970998+0000","last_clean":"2024-09-05T13:11:43.970998+0000","last_became_active":"2024-09-05T13:11:29.547775+0000","last_became_peered":"2024-09-05T13:11:29.547775+0000","last_unstale":"2024-09-05T13:11:43.970998+0000","last_undegraded":"2024-09-05T13:11:43.970998+0000","last_fullsized":"2024-09-05T13:11:43.970998+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:37:50.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.960199+0000","last_change":"2024-09-05T13:11:42.960199+0000","last_active":"2024-09-05T13:11:42.960199+0000","last_peered":"2024-09-05T13:11:42.508067+0000","last_clean":"2024-09-05T13:11:42.508067+0000","last_became_active":"2024-09-05T13:11:40.944779+0000","last_became_peered":"2024-09-05T13:11:40.944779+0000","last_unstale":"2024-09-05T13:11:42.960199+0000","last_undegraded":"2024-09-05T13:11:42.960199+0000","last_fullsized":"2024-09-05T13:11:42.960199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509567+0000","last_change":"2024-09-05T13:11:42.959022+0000","last_active":"2024-09-05T13:11:28.526981+0000","last_peered":"2024-09-05T13:11:28.526981+0000","last_clean":"2024-09-05T13:11:28.526981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509567+0000","last_undegraded":"2024-09-05T13:11:43.509567+0000","last_fullsized":"2024-09-05T13:11:43.509567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.971432+0000","last_change":"2024-09-05T13:11:40.946062+0000","last_active":"2024-09-05T13:11:43.971432+0000","last_peered":"2024-09-05T13:11:43.971432+0000","last_clean":"2024-09-05T13:11:43.971432+0000","last_became_active":"2024-09-05T13:11:40.945808+0000","last_became_peered":"2024-09-05T13:11:40.945808+0000","last_unstale":"2024-09-05T13:11:43.971432+0000","last_undegraded":"2024-09-05T13:11:43.971432+0000","last_fullsized":"2024-09-05T13:11:43.971432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:00:42.301934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.971525+0000","last_change":"2024-09-05T13:11:31.561639+0000","last_active":"2024-09-05T13:11:43.971525+0000","last_peered":"2024-09-05T13:11:43.971525+0000","last_clean":"2024-09-05T13:11:43.971525+0000","last_became_active":"2024-09-05T13:11:29.551153+0000","last_became_peered":"2024-09-05T13:11:29.551153+0000","last_unstale":"2024-09-05T13:11:43.971525+0000","last_undegraded":"2024-09-05T13:11:43.971525+0000","last_fullsized":"2024-09-05T13:11:43.971525+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:15:21.585273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-05T13:11:43.970676+0000","last_change":"2024-09-05T13:11:42.960916+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.970676+0000","last_undegraded":"2024-09-05T13:11:43.970676+0000","last_fullsized":"2024-09-05T13:11:43.970676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.961034+0000","last_change":"2024-09-05T13:11:42.961034+0000","last_active":"2024-09-05T13:11:42.961034+0000","last_peered":"2024-09-05T13:11:41.950057+0000","last_clean":"2024-09-05T13:11:41.950057+0000","last_became_active":"2024-09-05T13:11:29.558411+0000","last_became_peered":"2024-09-05T13:11:29.558411+0000","last_unstale":"2024-09-05T13:11:42.961034+0000","last_undegraded":"2024-09-05T13:11:42.961034+0000","last_fullsized":"2024-09-05T13:11:42.961034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-05T13:11:43.970869+0000","last_change":"2024-09-05T13:11:42.960264+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.970869+0000","last_undegraded":"2024-09-05T13:11:43.970869+0000","last_fullsized":"2024-09-05T13:11:43.970869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970201+0000","last_change":"2024-09-05T13:11:40.945347+0000","last_active":"2024-09-05T13:11:43.970201+0000","last_peered":"2024-09-05T13:11:43.970201+0000","last_clean":"2024-09-05T13:11:43.970201+0000","last_became_active":"2024-09-05T13:11:40.943950+0000","last_became_peered":"2024-09-05T13:11:40.943950+0000","last_unstale":"2024-09-05T13:11:43.970201+0000","last_undegraded":"2024-09-05T13:11:43.970201+0000","last_fullsized":"2024-09-05T13:11:43.970201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:31:54.180891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970048+0000","last_change":"2024-09-05T13:11:31.562474+0000","last_active":"2024-09-05T13:11:43.970048+0000","last_peered":"2024-09-05T13:11:43.970048+0000","last_clean":"2024-09-05T13:11:43.970048+0000","last_became_active":"2024-09-05T13:11:29.550485+0000","last_became_peered":"2024-09-05T13:11:29.550485+0000","last_unstale":"2024-09-05T13:11:43.970048+0000","last_undegraded":"2024-09-05T13:11:43.970048+0000","last_fullsized":"2024-09-05T13:11:43.970048+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:42:09.992680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969735+0000","last_change":"2024-09-05T13:11:40.944245+0000","last_active":"2024-09-05T13:11:43.969735+0000","last_peered":"2024-09-05T13:11:43.969735+0000","last_clean":"2024-09-05T13:11:43.969735+0000","last_became_active":"2024-09-05T13:11:40.943988+0000","last_became_peered":"2024-09-05T13:11:40.943988+0000","last_unstale":"2024-09-05T13:11:43.969735+0000","last_undegraded":"2024-09-05T13:11:43.969735+0000","last_fullsized":"2024-09-05T13:11:43.969735+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:57:41.082519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509601+0000","last_change":"2024-09-05T13:11:42.959358+0000","last_active":"2024-09-05T13:11:28.526981+0000","last_peered":"2024-09-05T13:11:28.526981+0000","last_clean":"2024-09-05T13:11:28.526981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509601+0000","last_undegraded":"2024-09-05T13:11:43.509601+0000","last_fullsized":"2024-09-05T13:11:43.509601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970030+0000","last_change":"2024-09-05T13:11:40.944217+0000","last_active":"2024-09-05T13:11:43.970030+0000","last_peered":"2024-09-05T13:11:43.970030+0000","last_clean":"2024-09-05T13:11:43.970030+0000","last_became_active":"2024-09-05T13:11:40.943942+0000","last_became_peered":"2024-09-05T13:11:40.943942+0000","last_unstale":"2024-09-05T13:11:43.970030+0000","last_undegraded":"2024-09-05T13:11:43.970030+0000","last_fullsized":"2024-09-05T13:11:43.970030+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:59.107892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969863+0000","last_change":"2024-09-05T13:11:31.561623+0000","last_active":"2024-09-05T13:11:43.969863+0000","last_peered":"2024-09-05T13:11:43.969863+0000","last_clean":"2024-09-05T13:11:43.969863+0000","last_became_active":"2024-09-05T13:11:29.550478+0000","last_became_peered":"2024-09-05T13:11:29.550478+0000","last_unstale":"2024-09-05T13:11:43.969863+0000","last_undegraded":"2024-09-05T13:11:43.969863+0000","last_fullsized":"2024-09-05T13:11:43.969863+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:20:35.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.961011+0000","last_change":"2024-09-05T13:11:42.961011+0000","last_active":"2024-09-05T13:11:42.961011+0000","last_peered":"2024-09-05T13:11:41.949917+0000","last_clean":"2024-09-05T13:11:41.949917+0000","last_became_active":"2024-09-05T13:11:29.558450+0000","last_became_peered":"2024-09-05T13:11:29.558450+0000","last_unstale":"2024-09-05T13:11:42.961011+0000","last_undegraded":"2024-09-05T13:11:42.961011+0000","last_fullsized":"2024-09-05T13:11:42.961011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970320+0000","last_change":"2024-09-05T13:11:40.945339+0000","last_active":"2024-09-05T13:11:43.970320+0000","last_peered":"2024-09-05T13:11:43.970320+0000","last_clean":"2024-09-05T13:11:43.970320+0000","last_became_active":"2024-09-05T13:11:40.945057+0000","last_became_peered":"2024-09-05T13:11:40.945057+0000","last_unstale":"2024-09-05T13:11:43.970320+0000","last_undegraded":"2024-09-05T13:11:43.970320+0000","last_fullsized":"2024-09-05T13:11:43.970320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:31:05.502438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":167936,"data_stored":43611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:11:47.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:11:47.720 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:11:47.731 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-05T13:11:47.501478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54392,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689288,"statfs":{"total":193273528320,"available":193217830912,"internally_reserved":0,"allocated":335872,"data_stored":87222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.553029"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969424+0000","last_change":"2024-09-05T13:11:40.946256+0000","last_active":"2024-09-05T13:11:43.969424+0000","last_peered":"2024-09-05T13:11:43.969424+0000","last_clean":"2024-09-05T13:11:43.969424+0000","last_became_active":"2024-09-05T13:11:40.945982+0000","last_became_peered":"2024-09-05T13:11:40.945982+0000","last_unstale":"2024-09-05T13:11:43.969424+0000","last_undegraded":"2024-09-05T13:11:43.969424+0000","last_fullsized":"2024-09-05T13:11:43.969424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:28:56.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509608+0000","last_change":"2024-09-05T13:11:42.959184+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509608+0000","last_undegraded":"2024-09-05T13:11:43.509608+0000","last_fullsized":"2024-09-05T13:11:43.509608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.960474+0000","last_change":"2024-09-05T13:11:42.960474+0000","last_active":"2024-09-05T13:11:42.960474+0000","last_peered":"2024-09-05T13:11:42.508089+0000","last_clean":"2024-09-05T13:11:42.508089+0000","last_became_active":"2024-09-05T13:11:40.944583+0000","last_became_peered":"2024-09-05T13:11:40.944583+0000","last_unstale":"2024-09-05T13:11:42.960474+0000","last_undegraded":"2024-09-05T13:11:42.960474+0000","last_fullsized":"2024-09-05T13:11:42.960474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509595+0000","last_change":"2024-09-05T13:11:42.959615+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509595+0000","last_undegraded":"2024-09-05T13:11:43.509595+0000","last_fullsized":"2024-09-05T13:11:43.509595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509587+0000","last_change":"2024-09-05T13:11:42.959048+0000","last_active":"2024-09-05T13:11:42.959048+0000","last_peered":"2024-09-05T13:11:41.951881+0000","last_clean":"2024-09-05T13:11:41.951881+0000","last_became_active":"2024-09-05T13:11:40.942058+0000","last_became_peered":"2024-09-05T13:11:40.942058+0000","last_unstale":"2024-09-05T13:11:43.509587+0000","last_undegraded":"2024-09-05T13:11:43.509587+0000","last_fullsized":"2024-09-05T13:11:43.509587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969835+0000","last_change":"2024-09-05T13:11:40.945249+0000","last_active":"2024-09-05T13:11:43.969835+0000","last_peered":"2024-09-05T13:11:43.969835+0000","last_clean":"2024-09-05T13:11:43.969835+0000","last_became_active":"2024-09-05T13:11:40.945009+0000","last_became_peered":"2024-09-05T13:11:40.945009+0000","last_unstale":"2024-09-05T13:11:43.969835+0000","last_undegraded":"2024-09-05T13:11:43.969835+0000","last_fullsized":"2024-09-05T13:11:43.969835+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:35.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970821+0000","last_change":"2024-09-05T13:11:40.944662+0000","last_active":"2024-09-05T13:11:43.970821+0000","last_peered":"2024-09-05T13:11:43.970821+0000","last_clean":"2024-09-05T13:11:43.970821+0000","last_became_active":"2024-09-05T13:11:40.944287+0000","last_became_peered":"2024-09-05T13:11:40.944287+0000","last_unstale":"2024-09-05T13:11:43.970821+0000","last_undegraded":"2024-09-05T13:11:43.970821+0000","last_fullsized":"2024-09-05T13:11:43.970821+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:15:15.352497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509544+0000","last_change":"2024-09-05T13:11:42.960433+0000","last_active":"2024-09-05T13:11:42.960433+0000","last_peered":"2024-09-05T13:11:41.951561+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:11:40.941932+0000","last_became_peered":"2024-09-05T13:11:40.941932+0000","last_unstale":"2024-09-05T13:11:43.509544+0000","last_undegraded":"2024-09-05T13:11:43.509544+0000","last_fullsized":"2024-09-05T13:11:43.509544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970998+0000","last_change":"2024-09-05T13:11:31.561573+0000","last_active":"2024-09-05T13:11:43.970998+0000","last_peered":"2024-09-05T13:11:43.970998+0000","last_clean":"2024-09-05T13:11:43.970998+0000","last_became_active":"2024-09-05T13:11:29.547775+0000","last_became_peered":"2024-09-05T13:11:29.547775+0000","last_unstale":"2024-09-05T13:11:43.970998+0000","last_undegraded":"2024-09-05T13:11:43.970998+0000","last_fullsized":"2024-09-05T13:11:43.970998+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:37:50.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.960199+0000","last_change":"2024-09-05T13:11:42.960199+0000","last_active":"2024-09-05T13:11:42.960199+0000","last_peered":"2024-09-05T13:11:42.508067+0000","last_clean":"2024-09-05T13:11:42.508067+0000","last_became_active":"2024-09-05T13:11:40.944779+0000","last_became_peered":"2024-09-05T13:11:40.944779+0000","last_unstale":"2024-09-05T13:11:42.960199+0000","last_undegraded":"2024-09-05T13:11:42.960199+0000","last_fullsized":"2024-09-05T13:11:42.960199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509567+0000","last_change":"2024-09-05T13:11:42.959022+0000","last_active":"2024-09-05T13:11:28.526981+0000","last_peered":"2024-09-05T13:11:28.526981+0000","last_clean":"2024-09-05T13:11:28.526981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509567+0000","last_undegraded":"2024-09-05T13:11:43.509567+0000","last_fullsized":"2024-09-05T13:11:43.509567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.971432+0000","last_change":"2024-09-05T13:11:40.946062+0000","last_active":"2024-09-05T13:11:43.971432+0000","last_peered":"2024-09-05T13:11:43.971432+0000","last_clean":"2024-09-05T13:11:43.971432+0000","last_became_active":"2024-09-05T13:11:40.945808+0000","last_became_peered":"2024-09-05T13:11:40.945808+0000","last_unstale":"2024-09-05T13:11:43.971432+0000","last_undegraded":"2024-09-05T13:11:43.971432+0000","last_fullsized":"2024-09-05T13:11:43.971432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:00:42.301934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.971525+0000","last_change":"2024-09-05T13:11:31.561639+0000","last_active":"2024-09-05T13:11:43.971525+0000","last_peered":"2024-09-05T13:11:43.971525+0000","last_clean":"2024-09-05T13:11:43.971525+0000","last_became_active":"2024-09-05T13:11:29.551153+0000","last_became_peered":"2024-09-05T13:11:29.551153+0000","last_unstale":"2024-09-05T13:11:43.971525+0000","last_undegraded":"2024-09-05T13:11:43.971525+0000","last_fullsized":"2024-09-05T13:11:43.971525+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:15:21.585273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-05T13:11:43.970676+0000","last_change":"2024-09-05T13:11:42.960916+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.970676+0000","last_undegraded":"2024-09-05T13:11:43.970676+0000","last_fullsized":"2024-09-05T13:11:43.970676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.961034+0000","last_change":"2024-09-05T13:11:42.961034+0000","last_active":"2024-09-05T13:11:42.961034+0000","last_peered":"2024-09-05T13:11:41.950057+0000","last_clean":"2024-09-05T13:11:41.950057+0000","last_became_active":"2024-09-05T13:11:29.558411+0000","last_became_peered":"2024-09-05T13:11:29.558411+0000","last_unstale":"2024-09-05T13:11:42.961034+0000","last_undegraded":"2024-09-05T13:11:42.961034+0000","last_fullsized":"2024-09-05T13:11:42.961034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-05T13:11:43.970869+0000","last_change":"2024-09-05T13:11:42.960264+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.970869+0000","last_undegraded":"2024-09-05T13:11:43.970869+0000","last_fullsized":"2024-09-05T13:11:43.970869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970201+0000","last_change":"2024-09-05T13:11:40.945347+0000","last_active":"2024-09-05T13:11:43.970201+0000","last_peered":"2024-09-05T13:11:43.970201+0000","last_clean":"2024-09-05T13:11:43.970201+0000","last_became_active":"2024-09-05T13:11:40.943950+0000","last_became_peered":"2024-09-05T13:11:40.943950+0000","last_unstale":"2024-09-05T13:11:43.970201+0000","last_undegraded":"2024-09-05T13:11:43.970201+0000","last_fullsized":"2024-09-05T13:11:43.970201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:31:54.180891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970048+0000","last_change":"2024-09-05T13:11:31.562474+0000","last_active":"2024-09-05T13:11:43.970048+0000","last_peered":"2024-09-05T13:11:43.970048+0000","last_clean":"2024-09-05T13:11:43.970048+0000","last_became_active":"2024-09-05T13:11:29.550485+0000","last_became_peered":"2024-09-05T13:11:29.550485+0000","last_unstale":"2024-09-05T13:11:43.970048+0000","last_undegraded":"2024-09-05T13:11:43.970048+0000","last_fullsized":"2024-09-05T13:11:43.970048+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:42:09.992680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969735+0000","last_change":"2024-09-05T13:11:40.944245+0000","last_active":"2024-09-05T13:11:43.969735+0000","last_peered":"2024-09-05T13:11:43.969735+0000","last_clean":"2024-09-05T13:11:43.969735+0000","last_became_active":"2024-09-05T13:11:40.943988+0000","last_became_peered":"2024-09-05T13:11:40.943988+0000","last_unstale":"2024-09-05T13:11:43.969735+0000","last_undegraded":"2024-09-05T13:11:43.969735+0000","last_fullsized":"2024-09-05T13:11:43.969735+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:57:41.082519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509601+0000","last_change":"2024-09-05T13:11:42.959358+0000","last_active":"2024-09-05T13:11:28.526981+0000","last_peered":"2024-09-05T13:11:28.526981+0000","last_clean":"2024-09-05T13:11:28.526981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509601+0000","last_undegraded":"2024-09-05T13:11:43.509601+0000","last_fullsized":"2024-09-05T13:11:43.509601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970030+0000","last_change":"2024-09-05T13:11:40.944217+0000","last_active":"2024-09-05T13:11:43.970030+0000","last_peered":"2024-09-05T13:11:43.970030+0000","last_clean":"2024-09-05T13:11:43.970030+0000","last_became_active":"2024-09-05T13:11:40.943942+0000","last_became_peered":"2024-09-05T13:11:40.943942+0000","last_unstale":"2024-09-05T13:11:43.970030+0000","last_undegraded":"2024-09-05T13:11:43.970030+0000","last_fullsized":"2024-09-05T13:11:43.970030+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:59.107892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969863+0000","last_change":"2024-09-05T13:11:31.561623+0000","last_active":"2024-09-05T13:11:43.969863+0000","last_peered":"2024-09-05T13:11:43.969863+0000","last_clean":"2024-09-05T13:11:43.969863+0000","last_became_active":"2024-09-05T13:11:29.550478+0000","last_became_peered":"2024-09-05T13:11:29.550478+0000","last_unstale":"2024-09-05T13:11:43.969863+0000","last_undegraded":"2024-09-05T13:11:43.969863+0000","last_fullsized":"2024-09-05T13:11:43.969863+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:20:35.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.961011+0000","last_change":"2024-09-05T13:11:42.961011+0000","last_active":"2024-09-05T13:11:42.961011+0000","last_peered":"2024-09-05T13:11:41.949917+0000","last_clean":"2024-09-05T13:11:41.949917+0000","last_became_active":"2024-09-05T13:11:29.558450+0000","last_became_peered":"2024-09-05T13:11:29.558450+0000","last_unstale":"2024-09-05T13:11:42.961011+0000","last_undegraded":"2024-09-05T13:11:42.961011+0000","last_fullsized":"2024-09-05T13:11:42.961011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970320+0000","last_change":"2024-09-05T13:11:40.945339+0000","last_active":"2024-09-05T13:11:43.970320+0000","last_peered":"2024-09-05T13:11:43.970320+0000","last_clean":"2024-09-05T13:11:43.970320+0000","last_became_active":"2024-09-05T13:11:40.945057+0000","last_became_peered":"2024-09-05T13:11:40.945057+0000","last_unstale":"2024-09-05T13:11:43.970320+0000","last_undegraded":"2024-09-05T13:11:43.970320+0000","last_fullsized":"2024-09-05T13:11:43.970320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:31:05.502438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":167936,"data_stored":43611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:11:47.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:11:48.014 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:48.015 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:11:48.027 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-05T13:11:47.501478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54392,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689288,"statfs":{"total":193273528320,"available":193217830912,"internally_reserved":0,"allocated":335872,"data_stored":87222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.553029"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969424+0000","last_change":"2024-09-05T13:11:40.946256+0000","last_active":"2024-09-05T13:11:43.969424+0000","last_peered":"2024-09-05T13:11:43.969424+0000","last_clean":"2024-09-05T13:11:43.969424+0000","last_became_active":"2024-09-05T13:11:40.945982+0000","last_became_peered":"2024-09-05T13:11:40.945982+0000","last_unstale":"2024-09-05T13:11:43.969424+0000","last_undegraded":"2024-09-05T13:11:43.969424+0000","last_fullsized":"2024-09-05T13:11:43.969424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:28:56.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509608+0000","last_change":"2024-09-05T13:11:42.959184+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509608+0000","last_undegraded":"2024-09-05T13:11:43.509608+0000","last_fullsized":"2024-09-05T13:11:43.509608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.960474+0000","last_change":"2024-09-05T13:11:42.960474+0000","last_active":"2024-09-05T13:11:42.960474+0000","last_peered":"2024-09-05T13:11:42.508089+0000","last_clean":"2024-09-05T13:11:42.508089+0000","last_became_active":"2024-09-05T13:11:40.944583+0000","last_became_peered":"2024-09-05T13:11:40.944583+0000","last_unstale":"2024-09-05T13:11:42.960474+0000","last_undegraded":"2024-09-05T13:11:42.960474+0000","last_fullsized":"2024-09-05T13:11:42.960474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509595+0000","last_change":"2024-09-05T13:11:42.959615+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509595+0000","last_undegraded":"2024-09-05T13:11:43.509595+0000","last_fullsized":"2024-09-05T13:11:43.509595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509587+0000","last_change":"2024-09-05T13:11:42.959048+0000","last_active":"2024-09-05T13:11:42.959048+0000","last_peered":"2024-09-05T13:11:41.951881+0000","last_clean":"2024-09-05T13:11:41.951881+0000","last_became_active":"2024-09-05T13:11:40.942058+0000","last_became_peered":"2024-09-05T13:11:40.942058+0000","last_unstale":"2024-09-05T13:11:43.509587+0000","last_undegraded":"2024-09-05T13:11:43.509587+0000","last_fullsized":"2024-09-05T13:11:43.509587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969835+0000","last_change":"2024-09-05T13:11:40.945249+0000","last_active":"2024-09-05T13:11:43.969835+0000","last_peered":"2024-09-05T13:11:43.969835+0000","last_clean":"2024-09-05T13:11:43.969835+0000","last_became_active":"2024-09-05T13:11:40.945009+0000","last_became_peered":"2024-09-05T13:11:40.945009+0000","last_unstale":"2024-09-05T13:11:43.969835+0000","last_undegraded":"2024-09-05T13:11:43.969835+0000","last_fullsized":"2024-09-05T13:11:43.969835+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:35.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970821+0000","last_change":"2024-09-05T13:11:40.944662+0000","last_active":"2024-09-05T13:11:43.970821+0000","last_peered":"2024-09-05T13:11:43.970821+0000","last_clean":"2024-09-05T13:11:43.970821+0000","last_became_active":"2024-09-05T13:11:40.944287+0000","last_became_peered":"2024-09-05T13:11:40.944287+0000","last_unstale":"2024-09-05T13:11:43.970821+0000","last_undegraded":"2024-09-05T13:11:43.970821+0000","last_fullsized":"2024-09-05T13:11:43.970821+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:15:15.352497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509544+0000","last_change":"2024-09-05T13:11:42.960433+0000","last_active":"2024-09-05T13:11:42.960433+0000","last_peered":"2024-09-05T13:11:41.951561+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:11:40.941932+0000","last_became_peered":"2024-09-05T13:11:40.941932+0000","last_unstale":"2024-09-05T13:11:43.509544+0000","last_undegraded":"2024-09-05T13:11:43.509544+0000","last_fullsized":"2024-09-05T13:11:43.509544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970998+0000","last_change":"2024-09-05T13:11:31.561573+0000","last_active":"2024-09-05T13:11:43.970998+0000","last_peered":"2024-09-05T13:11:43.970998+0000","last_clean":"2024-09-05T13:11:43.970998+0000","last_became_active":"2024-09-05T13:11:29.547775+0000","last_became_peered":"2024-09-05T13:11:29.547775+0000","last_unstale":"2024-09-05T13:11:43.970998+0000","last_undegraded":"2024-09-05T13:11:43.970998+0000","last_fullsized":"2024-09-05T13:11:43.970998+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:37:50.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.960199+0000","last_change":"2024-09-05T13:11:42.960199+0000","last_active":"2024-09-05T13:11:42.960199+0000","last_peered":"2024-09-05T13:11:42.508067+0000","last_clean":"2024-09-05T13:11:42.508067+0000","last_became_active":"2024-09-05T13:11:40.944779+0000","last_became_peered":"2024-09-05T13:11:40.944779+0000","last_unstale":"2024-09-05T13:11:42.960199+0000","last_undegraded":"2024-09-05T13:11:42.960199+0000","last_fullsized":"2024-09-05T13:11:42.960199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509567+0000","last_change":"2024-09-05T13:11:42.959022+0000","last_active":"2024-09-05T13:11:28.526981+0000","last_peered":"2024-09-05T13:11:28.526981+0000","last_clean":"2024-09-05T13:11:28.526981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509567+0000","last_undegraded":"2024-09-05T13:11:43.509567+0000","last_fullsized":"2024-09-05T13:11:43.509567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.971432+0000","last_change":"2024-09-05T13:11:40.946062+0000","last_active":"2024-09-05T13:11:43.971432+0000","last_peered":"2024-09-05T13:11:43.971432+0000","last_clean":"2024-09-05T13:11:43.971432+0000","last_became_active":"2024-09-05T13:11:40.945808+0000","last_became_peered":"2024-09-05T13:11:40.945808+0000","last_unstale":"2024-09-05T13:11:43.971432+0000","last_undegraded":"2024-09-05T13:11:43.971432+0000","last_fullsized":"2024-09-05T13:11:43.971432+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:00:42.301934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.971525+0000","last_change":"2024-09-05T13:11:31.561639+0000","last_active":"2024-09-05T13:11:43.971525+0000","last_peered":"2024-09-05T13:11:43.971525+0000","last_clean":"2024-09-05T13:11:43.971525+0000","last_became_active":"2024-09-05T13:11:29.551153+0000","last_became_peered":"2024-09-05T13:11:29.551153+0000","last_unstale":"2024-09-05T13:11:43.971525+0000","last_undegraded":"2024-09-05T13:11:43.971525+0000","last_fullsized":"2024-09-05T13:11:43.971525+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:15:21.585273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-05T13:11:43.970676+0000","last_change":"2024-09-05T13:11:42.960916+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.970676+0000","last_undegraded":"2024-09-05T13:11:43.970676+0000","last_fullsized":"2024-09-05T13:11:43.970676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.961034+0000","last_change":"2024-09-05T13:11:42.961034+0000","last_active":"2024-09-05T13:11:42.961034+0000","last_peered":"2024-09-05T13:11:41.950057+0000","last_clean":"2024-09-05T13:11:41.950057+0000","last_became_active":"2024-09-05T13:11:29.558411+0000","last_became_peered":"2024-09-05T13:11:29.558411+0000","last_unstale":"2024-09-05T13:11:42.961034+0000","last_undegraded":"2024-09-05T13:11:42.961034+0000","last_fullsized":"2024-09-05T13:11:42.961034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-05T13:11:43.970869+0000","last_change":"2024-09-05T13:11:42.960264+0000","last_active":"2024-09-05T13:11:39.923202+0000","last_peered":"2024-09-05T13:11:39.923202+0000","last_clean":"2024-09-05T13:11:39.923202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.970869+0000","last_undegraded":"2024-09-05T13:11:43.970869+0000","last_fullsized":"2024-09-05T13:11:43.970869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970201+0000","last_change":"2024-09-05T13:11:40.945347+0000","last_active":"2024-09-05T13:11:43.970201+0000","last_peered":"2024-09-05T13:11:43.970201+0000","last_clean":"2024-09-05T13:11:43.970201+0000","last_became_active":"2024-09-05T13:11:40.943950+0000","last_became_peered":"2024-09-05T13:11:40.943950+0000","last_unstale":"2024-09-05T13:11:43.970201+0000","last_undegraded":"2024-09-05T13:11:43.970201+0000","last_fullsized":"2024-09-05T13:11:43.970201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:31:54.180891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970048+0000","last_change":"2024-09-05T13:11:31.562474+0000","last_active":"2024-09-05T13:11:43.970048+0000","last_peered":"2024-09-05T13:11:43.970048+0000","last_clean":"2024-09-05T13:11:43.970048+0000","last_became_active":"2024-09-05T13:11:29.550485+0000","last_became_peered":"2024-09-05T13:11:29.550485+0000","last_unstale":"2024-09-05T13:11:43.970048+0000","last_undegraded":"2024-09-05T13:11:43.970048+0000","last_fullsized":"2024-09-05T13:11:43.970048+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:42:09.992680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969735+0000","last_change":"2024-09-05T13:11:40.944245+0000","last_active":"2024-09-05T13:11:43.969735+0000","last_peered":"2024-09-05T13:11:43.969735+0000","last_clean":"2024-09-05T13:11:43.969735+0000","last_became_active":"2024-09-05T13:11:40.943988+0000","last_became_peered":"2024-09-05T13:11:40.943988+0000","last_unstale":"2024-09-05T13:11:43.969735+0000","last_undegraded":"2024-09-05T13:11:43.969735+0000","last_fullsized":"2024-09-05T13:11:43.969735+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:57:41.082519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:43.509601+0000","last_change":"2024-09-05T13:11:42.959358+0000","last_active":"2024-09-05T13:11:28.526981+0000","last_peered":"2024-09-05T13:11:28.526981+0000","last_clean":"2024-09-05T13:11:28.526981+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-05T13:11:43.509601+0000","last_undegraded":"2024-09-05T13:11:43.509601+0000","last_fullsized":"2024-09-05T13:11:43.509601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970030+0000","last_change":"2024-09-05T13:11:40.944217+0000","last_active":"2024-09-05T13:11:43.970030+0000","last_peered":"2024-09-05T13:11:43.970030+0000","last_clean":"2024-09-05T13:11:43.970030+0000","last_became_active":"2024-09-05T13:11:40.943942+0000","last_became_peered":"2024-09-05T13:11:40.943942+0000","last_unstale":"2024-09-05T13:11:43.970030+0000","last_undegraded":"2024-09-05T13:11:43.970030+0000","last_fullsized":"2024-09-05T13:11:43.970030+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:59.107892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.969863+0000","last_change":"2024-09-05T13:11:31.561623+0000","last_active":"2024-09-05T13:11:43.969863+0000","last_peered":"2024-09-05T13:11:43.969863+0000","last_clean":"2024-09-05T13:11:43.969863+0000","last_became_active":"2024-09-05T13:11:29.550478+0000","last_became_peered":"2024-09-05T13:11:29.550478+0000","last_unstale":"2024-09-05T13:11:43.969863+0000","last_undegraded":"2024-09-05T13:11:43.969863+0000","last_fullsized":"2024-09-05T13:11:43.969863+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:20:35.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-05T13:11:42.961011+0000","last_change":"2024-09-05T13:11:42.961011+0000","last_active":"2024-09-05T13:11:42.961011+0000","last_peered":"2024-09-05T13:11:41.949917+0000","last_clean":"2024-09-05T13:11:41.949917+0000","last_became_active":"2024-09-05T13:11:29.558450+0000","last_became_peered":"2024-09-05T13:11:29.558450+0000","last_unstale":"2024-09-05T13:11:42.961011+0000","last_undegraded":"2024-09-05T13:11:42.961011+0000","last_fullsized":"2024-09-05T13:11:42.961011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-05T13:11:43.970320+0000","last_change":"2024-09-05T13:11:40.945339+0000","last_active":"2024-09-05T13:11:43.970320+0000","last_peered":"2024-09-05T13:11:43.970320+0000","last_clean":"2024-09-05T13:11:43.970320+0000","last_became_active":"2024-09-05T13:11:40.945057+0000","last_became_peered":"2024-09-05T13:11:40.945057+0000","last_unstale":"2024-09-05T13:11:43.970320+0000","last_undegraded":"2024-09-05T13:11:43.970320+0000","last_fullsized":"2024-09-05T13:11:43.970320+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:31:05.502438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":167936,"data_stored":43611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:11:51.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:11:51.318 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:11:51.319 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:11:51.330 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-05T13:11:49.501735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54436,"kb_used_data":368,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689244,"statfs":{"total":193273528320,"available":193217785856,"internally_reserved":0,"allocated":376832,"data_stored":99172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.543998"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.972594+0000","last_change":"2024-09-05T13:11:40.946256+0000","last_active":"2024-09-05T13:11:44.972594+0000","last_peered":"2024-09-05T13:11:44.972594+0000","last_clean":"2024-09-05T13:11:44.972594+0000","last_became_active":"2024-09-05T13:11:40.945982+0000","last_became_peered":"2024-09-05T13:11:40.945982+0000","last_unstale":"2024-09-05T13:11:44.972594+0000","last_undegraded":"2024-09-05T13:11:44.972594+0000","last_fullsized":"2024-09-05T13:11:44.972594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:28:56.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.977032+0000","last_change":"2024-09-05T13:11:44.976949+0000","last_active":"2024-09-05T13:11:44.977032+0000","last_peered":"2024-09-05T13:11:44.977032+0000","last_clean":"2024-09-05T13:11:44.977032+0000","last_became_active":"2024-09-05T13:11:44.976653+0000","last_became_peered":"2024-09-05T13:11:44.976653+0000","last_unstale":"2024-09-05T13:11:44.977032+0000","last_undegraded":"2024-09-05T13:11:44.977032+0000","last_fullsized":"2024-09-05T13:11:44.977032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:29:49.973652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.979992+0000","last_change":"2024-09-05T13:11:44.979907+0000","last_active":"2024-09-05T13:11:44.979992+0000","last_peered":"2024-09-05T13:11:44.979992+0000","last_clean":"2024-09-05T13:11:44.979992+0000","last_became_active":"2024-09-05T13:11:44.979634+0000","last_became_peered":"2024-09-05T13:11:44.979634+0000","last_unstale":"2024-09-05T13:11:44.979992+0000","last_undegraded":"2024-09-05T13:11:44.979992+0000","last_fullsized":"2024-09-05T13:11:44.979992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:08.431440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.977030+0000","last_change":"2024-09-05T13:11:44.976942+0000","last_active":"2024-09-05T13:11:44.977030+0000","last_peered":"2024-09-05T13:11:44.977030+0000","last_clean":"2024-09-05T13:11:44.977030+0000","last_became_active":"2024-09-05T13:11:44.976658+0000","last_became_peered":"2024-09-05T13:11:44.976658+0000","last_unstale":"2024-09-05T13:11:44.977030+0000","last_undegraded":"2024-09-05T13:11:44.977030+0000","last_fullsized":"2024-09-05T13:11:44.977030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:29:37.569867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.977248+0000","last_change":"2024-09-05T13:11:44.977161+0000","last_active":"2024-09-05T13:11:44.977248+0000","last_peered":"2024-09-05T13:11:44.977248+0000","last_clean":"2024-09-05T13:11:44.977248+0000","last_became_active":"2024-09-05T13:11:44.976876+0000","last_became_peered":"2024-09-05T13:11:44.976876+0000","last_unstale":"2024-09-05T13:11:44.977248+0000","last_undegraded":"2024-09-05T13:11:44.977248+0000","last_fullsized":"2024-09-05T13:11:44.977248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:33:25.902164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.972477+0000","last_change":"2024-09-05T13:11:40.945249+0000","last_active":"2024-09-05T13:11:44.972477+0000","last_peered":"2024-09-05T13:11:44.972477+0000","last_clean":"2024-09-05T13:11:44.972477+0000","last_became_active":"2024-09-05T13:11:40.945009+0000","last_became_peered":"2024-09-05T13:11:40.945009+0000","last_unstale":"2024-09-05T13:11:44.972477+0000","last_undegraded":"2024-09-05T13:11:44.972477+0000","last_fullsized":"2024-09-05T13:11:44.972477+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:35.731593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.973530+0000","last_change":"2024-09-05T13:11:40.944662+0000","last_active":"2024-09-05T13:11:44.973530+0000","last_peered":"2024-09-05T13:11:44.973530+0000","last_clean":"2024-09-05T13:11:44.973530+0000","last_became_active":"2024-09-05T13:11:40.944287+0000","last_became_peered":"2024-09-05T13:11:40.944287+0000","last_unstale":"2024-09-05T13:11:44.973530+0000","last_undegraded":"2024-09-05T13:11:44.973530+0000","last_fullsized":"2024-09-05T13:11:44.973530+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:15:15.352497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.975533+0000","last_change":"2024-09-05T13:11:44.975477+0000","last_active":"2024-09-05T13:11:44.975533+0000","last_peered":"2024-09-05T13:11:44.975533+0000","last_clean":"2024-09-05T13:11:44.975533+0000","last_became_active":"2024-09-05T13:11:44.975266+0000","last_became_peered":"2024-09-05T13:11:44.975266+0000","last_unstale":"2024-09-05T13:11:44.975533+0000","last_undegraded":"2024-09-05T13:11:44.975533+0000","last_fullsized":"2024-09-05T13:11:44.975533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:57:27.131488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.972964+0000","last_change":"2024-09-05T13:11:31.561573+0000","last_active":"2024-09-05T13:11:44.972964+0000","last_peered":"2024-09-05T13:11:44.972964+0000","last_clean":"2024-09-05T13:11:44.972964+0000","last_became_active":"2024-09-05T13:11:29.547775+0000","last_became_peered":"2024-09-05T13:11:29.547775+0000","last_unstale":"2024-09-05T13:11:44.972964+0000","last_undegraded":"2024-09-05T13:11:44.972964+0000","last_fullsized":"2024-09-05T13:11:44.972964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:37:50.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.979904+0000","last_change":"2024-09-05T13:11:44.979810+0000","last_active":"2024-09-05T13:11:44.979904+0000","last_peered":"2024-09-05T13:11:44.979904+0000","last_clean":"2024-09-05T13:11:44.979904+0000","last_became_active":"2024-09-05T13:11:44.979545+0000","last_became_peered":"2024-09-05T13:11:44.979545+0000","last_unstale":"2024-09-05T13:11:44.979904+0000","last_undegraded":"2024-09-05T13:11:44.979904+0000","last_fullsized":"2024-09-05T13:11:44.979904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:26:44.990879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.976309+0000","last_change":"2024-09-05T13:11:44.976234+0000","last_active":"2024-09-05T13:11:44.976309+0000","last_peered":"2024-09-05T13:11:44.976309+0000","last_clean":"2024-09-05T13:11:44.976309+0000","last_became_active":"2024-09-05T13:11:44.976008+0000","last_became_peered":"2024-09-05T13:11:44.976008+0000","last_unstale":"2024-09-05T13:11:44.976309+0000","last_undegraded":"2024-09-05T13:11:44.976309+0000","last_fullsized":"2024-09-05T13:11:44.976309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:30.350581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.972753+0000","last_change":"2024-09-05T13:11:40.946062+0000","last_active":"2024-09-05T13:11:44.972753+0000","last_peered":"2024-09-05T13:11:44.972753+0000","last_clean":"2024-09-05T13:11:44.972753+0000","last_became_active":"2024-09-05T13:11:40.945808+0000","last_became_peered":"2024-09-05T13:11:40.945808+0000","last_unstale":"2024-09-05T13:11:44.972753+0000","last_undegraded":"2024-09-05T13:11:44.972753+0000","last_fullsized":"2024-09-05T13:11:44.972753+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:00:42.301934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":36,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.972822+0000","last_change":"2024-09-05T13:11:31.561639+0000","last_active":"2024-09-05T13:11:44.972822+0000","last_peered":"2024-09-05T13:11:44.972822+0000","last_clean":"2024-09-05T13:11:44.972822+0000","last_became_active":"2024-09-05T13:11:29.551153+0000","last_became_peered":"2024-09-05T13:11:29.551153+0000","last_unstale":"2024-09-05T13:11:44.972822+0000","last_undegraded":"2024-09-05T13:11:44.972822+0000","last_fullsized":"2024-09-05T13:11:44.972822+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:15:21.585273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.980456+0000","last_change":"2024-09-05T13:11:44.980366+0000","last_active":"2024-09-05T13:11:44.980456+0000","last_peered":"2024-09-05T13:11:44.980456+0000","last_clean":"2024-09-05T13:11:44.980456+0000","last_became_active":"2024-09-05T13:11:44.980022+0000","last_became_peered":"2024-09-05T13:11:44.980022+0000","last_unstale":"2024-09-05T13:11:44.980456+0000","last_undegraded":"2024-09-05T13:11:44.980456+0000","last_fullsized":"2024-09-05T13:11:44.980456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:38:43.862386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":44,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.983026+0000","last_change":"2024-09-05T13:11:44.982964+0000","last_active":"2024-09-05T13:11:44.983026+0000","last_peered":"2024-09-05T13:11:44.983026+0000","last_clean":"2024-09-05T13:11:44.983026+0000","last_became_active":"2024-09-05T13:11:44.980089+0000","last_became_peered":"2024-09-05T13:11:44.980089+0000","last_unstale":"2024-09-05T13:11:44.983026+0000","last_undegraded":"2024-09-05T13:11:44.983026+0000","last_fullsized":"2024-09-05T13:11:44.983026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:53:13.974526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.979945+0000","last_change":"2024-09-05T13:11:44.979851+0000","last_active":"2024-09-05T13:11:44.979945+0000","last_peered":"2024-09-05T13:11:44.979945+0000","last_clean":"2024-09-05T13:11:44.979945+0000","last_became_active":"2024-09-05T13:11:44.979581+0000","last_became_peered":"2024-09-05T13:11:44.979581+0000","last_unstale":"2024-09-05T13:11:44.979945+0000","last_undegraded":"2024-09-05T13:11:44.979945+0000","last_fullsized":"2024-09-05T13:11:44.979945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:23:20.814542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.973217+0000","last_change":"2024-09-05T13:11:40.945347+0000","last_active":"2024-09-05T13:11:44.973217+0000","last_peered":"2024-09-05T13:11:44.973217+0000","last_clean":"2024-09-05T13:11:44.973217+0000","last_became_active":"2024-09-05T13:11:40.943950+0000","last_became_peered":"2024-09-05T13:11:40.943950+0000","last_unstale":"2024-09-05T13:11:44.973217+0000","last_undegraded":"2024-09-05T13:11:44.973217+0000","last_fullsized":"2024-09-05T13:11:44.973217+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:31:54.180891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.973052+0000","last_change":"2024-09-05T13:11:31.562474+0000","last_active":"2024-09-05T13:11:44.973052+0000","last_peered":"2024-09-05T13:11:44.973052+0000","last_clean":"2024-09-05T13:11:44.973052+0000","last_became_active":"2024-09-05T13:11:29.550485+0000","last_became_peered":"2024-09-05T13:11:29.550485+0000","last_unstale":"2024-09-05T13:11:44.973052+0000","last_undegraded":"2024-09-05T13:11:44.973052+0000","last_fullsized":"2024-09-05T13:11:44.973052+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:42:09.992680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.973128+0000","last_change":"2024-09-05T13:11:40.944245+0000","last_active":"2024-09-05T13:11:44.973128+0000","last_peered":"2024-09-05T13:11:44.973128+0000","last_clean":"2024-09-05T13:11:44.973128+0000","last_became_active":"2024-09-05T13:11:40.943988+0000","last_became_peered":"2024-09-05T13:11:40.943988+0000","last_unstale":"2024-09-05T13:11:44.973128+0000","last_undegraded":"2024-09-05T13:11:44.973128+0000","last_fullsized":"2024-09-05T13:11:44.973128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:57:41.082519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.977074+0000","last_change":"2024-09-05T13:11:44.976989+0000","last_active":"2024-09-05T13:11:44.977074+0000","last_peered":"2024-09-05T13:11:44.977074+0000","last_clean":"2024-09-05T13:11:44.977074+0000","last_became_active":"2024-09-05T13:11:44.976630+0000","last_became_peered":"2024-09-05T13:11:44.976630+0000","last_unstale":"2024-09-05T13:11:44.977074+0000","last_undegraded":"2024-09-05T13:11:44.977074+0000","last_fullsized":"2024-09-05T13:11:44.977074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:12:53.098763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.973299+0000","last_change":"2024-09-05T13:11:40.944217+0000","last_active":"2024-09-05T13:11:44.973299+0000","last_peered":"2024-09-05T13:11:44.973299+0000","last_clean":"2024-09-05T13:11:44.973299+0000","last_became_active":"2024-09-05T13:11:40.943942+0000","last_became_peered":"2024-09-05T13:11:40.943942+0000","last_unstale":"2024-09-05T13:11:44.973299+0000","last_undegraded":"2024-09-05T13:11:44.973299+0000","last_fullsized":"2024-09-05T13:11:44.973299+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:14:59.107892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.973172+0000","last_change":"2024-09-05T13:11:31.561623+0000","last_active":"2024-09-05T13:11:44.973172+0000","last_peered":"2024-09-05T13:11:44.973172+0000","last_clean":"2024-09-05T13:11:44.973172+0000","last_became_active":"2024-09-05T13:11:29.550478+0000","last_became_peered":"2024-09-05T13:11:29.550478+0000","last_unstale":"2024-09-05T13:11:44.973172+0000","last_undegraded":"2024-09-05T13:11:44.973172+0000","last_fullsized":"2024-09-05T13:11:44.973172+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:20:35.263529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.979973+0000","last_change":"2024-09-05T13:11:44.979879+0000","last_active":"2024-09-05T13:11:44.979973+0000","last_peered":"2024-09-05T13:11:44.979973+0000","last_clean":"2024-09-05T13:11:44.979973+0000","last_became_active":"2024-09-05T13:11:44.979591+0000","last_became_peered":"2024-09-05T13:11:44.979591+0000","last_unstale":"2024-09-05T13:11:44.979973+0000","last_undegraded":"2024-09-05T13:11:44.979973+0000","last_fullsized":"2024-09-05T13:11:44.979973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:49:37.030866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-05T13:11:44.972988+0000","last_change":"2024-09-05T13:11:40.945339+0000","last_active":"2024-09-05T13:11:44.972988+0000","last_peered":"2024-09-05T13:11:44.972988+0000","last_clean":"2024-09-05T13:11:44.972988+0000","last_became_active":"2024-09-05T13:11:40.945057+0000","last_became_peered":"2024-09-05T13:11:40.945057+0000","last_unstale":"2024-09-05T13:11:44.972988+0000","last_undegraded":"2024-09-05T13:11:44.972988+0000","last_fullsized":"2024-09-05T13:11:44.972988+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:31:05.502438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705675,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":49586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":49586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:11:51.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-05T13:11:51.331 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-05T13:11:51.331 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-05T13:11:51.381 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-05T13:11:51.435 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-05T13:11:51.484 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_1 2024-09-05T13:11:51.484 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_1 2024-09-05T13:11:51.531 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-05T13:11:51.531 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-05T13:11:51.579 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-05T13:11:51.580 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-05T13:11:51.627 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_2 2024-09-05T13:11:51.628 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_2 2024-09-05T13:11:51.675 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-05T13:11:51.675 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-05T13:11:51.723 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-05T13:11:51.723 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-05T13:11:51.771 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_3 2024-09-05T13:11:51.771 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_3 2024-09-05T13:11:51.819 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-05T13:11:51.820 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-05T13:11:51.868 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-05T13:11:51.868 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-05T13:11:51.916 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_4 2024-09-05T13:11:51.916 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_4 2024-09-05T13:11:51.963 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-05T13:11:51.963 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-05T13:11:52.012 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-05T13:11:52.012 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-05T13:11:52.060 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_5 2024-09-05T13:11:52.060 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_5 2024-09-05T13:11:52.106 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-05T13:11:52.106 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-05T13:11:52.157 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-05T13:11:52.157 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-05T13:11:52.205 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_6 2024-09-05T13:11:52.205 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_6 2024-09-05T13:11:52.251 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-05T13:11:52.252 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-05T13:11:52.300 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-05T13:11:52.300 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-05T13:11:52.347 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_7 2024-09-05T13:11:52.348 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_7 2024-09-05T13:11:52.394 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-05T13:11:52.395 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-05T13:11:52.442 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-05T13:11:52.443 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-05T13:11:52.490 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_8 2024-09-05T13:11:52.491 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_8 2024-09-05T13:11:52.537 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-05T13:11:52.538 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-05T13:11:52.586 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-05T13:11:52.586 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-05T13:11:52.635 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_9 2024-09-05T13:11:52.636 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_9 2024-09-05T13:11:52.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-05T13:11:52.802 INFO:teuthology.orchestra.run.smithi086.stdout:{} 2024-09-05T13:11:52.802 INFO:teuthology.orchestra.run.smithi086.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-05T13:11:52.812 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-05T13:11:52.813 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:11:52.841 INFO:tasks.ceph.osd.0:Stopped 2024-09-05T13:11:52.841 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-05T13:11:53.182 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:11:53.181+0000 7f9a87bca640 -1 mon.a@0(leader).osd e22 definitely_dead 0 2024-09-05T13:11:53.182 INFO:teuthology.orchestra.run.smithi086.stderr:osd.0 is already down. 2024-09-05T13:11:53.195 INFO:tasks.util.rados:rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-05T13:11:53.195 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-05T13:11:54.010 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-05T13:11:54.010 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-05T13:11:54.060 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-05T13:11:54.060 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-05T13:11:54.109 INFO:tasks.util.rados:rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-05T13:11:54.109 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-05T13:11:54.158 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-05T13:11:54.158 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-05T13:11:54.206 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-05T13:11:54.206 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-05T13:11:54.253 INFO:tasks.util.rados:rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-05T13:11:54.254 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-05T13:11:54.302 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-05T13:11:54.302 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-05T13:11:54.349 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-05T13:11:54.350 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-05T13:11:54.397 INFO:tasks.util.rados:rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-05T13:11:54.397 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-05T13:11:54.445 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-05T13:11:54.445 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-05T13:11:54.493 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-05T13:11:54.493 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-05T13:11:54.541 INFO:tasks.util.rados:rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-05T13:11:54.541 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-05T13:11:54.589 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-05T13:11:54.590 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-05T13:11:54.637 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-05T13:11:54.637 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-05T13:11:54.685 INFO:tasks.util.rados:rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-05T13:11:54.686 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-05T13:11:54.733 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-05T13:11:54.733 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-05T13:11:54.781 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-05T13:11:54.781 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-05T13:11:54.829 INFO:tasks.util.rados:rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-05T13:11:54.829 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-05T13:11:54.877 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-05T13:11:54.877 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-05T13:11:54.924 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-05T13:11:54.925 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-05T13:11:54.972 INFO:tasks.util.rados:rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-05T13:11:54.972 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-05T13:11:55.021 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-05T13:11:55.021 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-05T13:11:55.069 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-05T13:11:55.069 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-05T13:11:55.116 INFO:tasks.util.rados:rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-05T13:11:55.117 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-05T13:11:55.165 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-05T13:11:55.165 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-05T13:11:55.212 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-05T13:11:55.213 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-05T13:11:55.260 INFO:tasks.rep_lost_unfound_delete:osd.0 command_args is foo 2024-09-05T13:11:55.260 INFO:tasks.rep_lost_unfound_delete:() 2024-09-05T13:11:55.260 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-05T13:11:55.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-05T13:11:55.262 INFO:tasks.ceph.osd.0:Started 2024-09-05T13:11:55.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:no valid command found; 10 closest matches: 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:0 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:1 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:abort 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:assert 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:config diff 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:config diff get 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:config get 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:config help [] 2024-09-05T13:11:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:config set ... 2024-09-05T13:11:55.361 INFO:teuthology.orchestra.run.smithi086.stderr:admin_socket: invalid command 2024-09-05T13:11:55.361 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-05T13:11:55.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-05T13:11:56.666 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:11:56.664+0000 7fc47bc11640 -1 Falling back to public interface 2024-09-05T13:12:00.208 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:00.207+0000 7fc47bc11640 -1 osd.0 21 log_to_monitors true 2024-09-05T13:12:00.363 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-05T13:12:00.462 INFO:teuthology.orchestra.run.smithi086.stderr:no valid command found; 10 closest matches: 2024-09-05T13:12:00.462 INFO:teuthology.orchestra.run.smithi086.stderr:0 2024-09-05T13:12:00.462 INFO:teuthology.orchestra.run.smithi086.stderr:1 2024-09-05T13:12:00.462 INFO:teuthology.orchestra.run.smithi086.stderr:2 2024-09-05T13:12:00.463 INFO:teuthology.orchestra.run.smithi086.stderr:abort 2024-09-05T13:12:00.463 INFO:teuthology.orchestra.run.smithi086.stderr:assert 2024-09-05T13:12:00.463 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs debug_inject_read_zeros 2024-09-05T13:12:00.463 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs files list 2024-09-05T13:12:00.463 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs stats 2024-09-05T13:12:00.463 INFO:teuthology.orchestra.run.smithi086.stderr:bluestore allocator dump block 2024-09-05T13:12:00.463 INFO:teuthology.orchestra.run.smithi086.stderr:bluestore allocator fragmentation block 2024-09-05T13:12:00.463 INFO:teuthology.orchestra.run.smithi086.stderr:admin_socket: invalid command 2024-09-05T13:12:00.464 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-05T13:12:00.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-05T13:12:01.070 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:01.069+0000 7fc473345640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T13:12:05.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-05T13:12:05.582 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:05.582 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-05T13:12:05.582 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-05T13:12:05.582 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:05.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-05T13:12:05.881 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:12:05.880+0000 7f9a87bca640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-05T13:12:05.881 INFO:teuthology.orchestra.run.smithi086.stderr:osd.0 is already in. 2024-09-05T13:12:05.894 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.0 to be up 2024-09-05T13:12:05.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T13:12:06.180 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:06.180 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:12:03.067653+0000","last_up_change":"2024-09-05T13:12:02.061731+0000","last_in_change":"2024-09-05T13:11:43.954391+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-05T13:11:28.452062+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-05T13:11:39.270087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":25,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6804","nonce":2531722421}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6805","nonce":2531722421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6807","nonce":2531722421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6806","nonce":2531722421}]},"public_addr":"172.21.15.86:6804/2531722421","cluster_addr":"172.21.15.86:6805/2531722421","heartbeat_back_addr":"172.21.15.86:6807/2531722421","heartbeat_front_addr":"172.21.15.86:6806/2531722421","state":["exists","up"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2999410110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2999410110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2999410110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2999410110}]},"public_addr":"172.21.15.86:6800/2999410110","cluster_addr":"172.21.15.86:6801/2999410110","heartbeat_back_addr":"172.21.15.86:6803/2999410110","heartbeat_front_addr":"172.21.15.86:6802/2999410110","state":["exists","up"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6808","nonce":184901110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6809","nonce":184901110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6811","nonce":184901110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6810","nonce":184901110}]},"public_addr":"172.21.15.86:6808/184901110","cluster_addr":"172.21.15.86:6809/184901110","heartbeat_back_addr":"172.21.15.86:6811/184901110","heartbeat_front_addr":"172.21.15.86:6810/184901110","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.643488+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.440515+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-05T13:11:21.020419+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:12:06.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.0 is up 2024-09-05T13:12:06.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-05T13:12:06.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-05T13:12:06.326 INFO:teuthology.orchestra.run.smithi086.stdout:107374182403 2024-09-05T13:12:06.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:06.327 INFO:teuthology.orchestra.run.smithi086.stdout:34359738382 2024-09-05T13:12:06.327 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:12:06.653 INFO:teuthology.orchestra.run.smithi086.stdout:34359738381 2024-09-05T13:12:06.654 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-09-05T13:12:06.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-05T13:12:06.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738382 got 34359738381 for osd.1 2024-09-05T13:12:07.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:07.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:12:07.971 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-09-05T13:12:07.975 INFO:teuthology.orchestra.run.smithi086.stdout:34359738381 2024-09-05T13:12:07.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-05T13:12:07.988 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738382 got 34359738381 for osd.1 2024-09-05T13:12:08.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:08.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-05T13:12:09.293 INFO:teuthology.orchestra.run.smithi086.stdout:107374182403 2024-09-05T13:12:09.294 INFO:teuthology.orchestra.run.smithi086.stdout:34359738382 2024-09-05T13:12:09.307 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738382 got 34359738382 for osd.1 2024-09-05T13:12:09.307 DEBUG:teuthology.parallel:result is None 2024-09-05T13:12:09.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182403 got 107374182403 for osd.0 2024-09-05T13:12:09.308 DEBUG:teuthology.parallel:result is None 2024-09-05T13:12:09.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-05T13:12:09.308 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:09.589 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:09.589 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:09.601 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-05T13:12:09.505267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55108,"kb_used_data":592,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688572,"statfs":{"total":193273528320,"available":193217097728,"internally_reserved":0,"allocated":606208,"data_stored":147528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6033,"internal_metadata":55765103},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001372"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.084510+0000","last_change":"2024-09-05T13:12:03.084422+0000","last_active":"2024-09-05T13:12:03.084510+0000","last_peered":"2024-09-05T13:12:03.084510+0000","last_clean":"2024-09-05T13:12:03.084510+0000","last_became_active":"2024-09-05T13:12:03.084138+0000","last_became_peered":"2024-09-05T13:12:03.084138+0000","last_unstale":"2024-09-05T13:12:03.084510+0000","last_undegraded":"2024-09-05T13:12:03.084510+0000","last_fullsized":"2024-09-05T13:12:03.084510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:05:50.999797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":25,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:03.083841+0000","last_change":"2024-09-05T13:12:03.088186+0000","last_active":"2024-09-05T13:12:03.083841+0000","last_peered":"2024-09-05T13:12:03.083841+0000","last_clean":"2024-09-05T13:11:51.865448+0000","last_became_active":"2024-09-05T13:12:03.083437+0000","last_became_peered":"2024-09-05T13:12:03.083437+0000","last_unstale":"2024-09-05T13:12:03.083841+0000","last_undegraded":"2024-09-05T13:12:03.072399+0000","last_fullsized":"2024-09-05T13:12:03.083841+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":44,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.085217+0000","last_change":"2024-09-05T13:12:03.085434+0000","last_active":"2024-09-05T13:12:03.085217+0000","last_peered":"2024-09-05T13:12:03.085217+0000","last_clean":"2024-09-05T13:11:52.391844+0000","last_became_active":"2024-09-05T13:12:03.085125+0000","last_became_peered":"2024-09-05T13:12:03.085125+0000","last_unstale":"2024-09-05T13:12:03.085217+0000","last_undegraded":"2024-09-05T13:12:03.075663+0000","last_fullsized":"2024-09-05T13:12:03.085217+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.083959+0000","last_change":"2024-09-05T13:12:03.083858+0000","last_active":"2024-09-05T13:12:03.083959+0000","last_peered":"2024-09-05T13:12:03.083959+0000","last_clean":"2024-09-05T13:12:03.083959+0000","last_became_active":"2024-09-05T13:12:03.083520+0000","last_became_peered":"2024-09-05T13:12:03.083520+0000","last_unstale":"2024-09-05T13:12:03.083959+0000","last_undegraded":"2024-09-05T13:12:03.083959+0000","last_fullsized":"2024-09-05T13:12:03.083959+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:25:31.985639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.083657+0000","last_change":"2024-09-05T13:12:03.084379+0000","last_active":"2024-09-05T13:12:03.083657+0000","last_peered":"2024-09-05T13:12:03.083657+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:03.083556+0000","last_became_peered":"2024-09-05T13:12:03.083556+0000","last_unstale":"2024-09-05T13:12:03.083657+0000","last_undegraded":"2024-09-05T13:12:03.073631+0000","last_fullsized":"2024-09-05T13:12:03.083657+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.085283+0000","last_change":"2024-09-05T13:12:03.085733+0000","last_active":"2024-09-05T13:12:03.085283+0000","last_peered":"2024-09-05T13:12:03.085283+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:03.085202+0000","last_became_peered":"2024-09-05T13:12:03.085202+0000","last_unstale":"2024-09-05T13:12:03.085283+0000","last_undegraded":"2024-09-05T13:12:03.076479+0000","last_fullsized":"2024-09-05T13:12:03.085283+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":40,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.085281+0000","last_change":"2024-09-05T13:12:03.085609+0000","last_active":"2024-09-05T13:12:03.085281+0000","last_peered":"2024-09-05T13:12:03.085281+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:03.085186+0000","last_became_peered":"2024-09-05T13:12:03.085186+0000","last_unstale":"2024-09-05T13:12:03.085281+0000","last_undegraded":"2024-09-05T13:12:03.077811+0000","last_fullsized":"2024-09-05T13:12:03.085281+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":33,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.083665+0000","last_change":"2024-09-05T13:12:03.083889+0000","last_active":"2024-09-05T13:12:03.083665+0000","last_peered":"2024-09-05T13:12:03.083665+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:03.083473+0000","last_became_peered":"2024-09-05T13:12:03.083473+0000","last_unstale":"2024-09-05T13:12:03.083665+0000","last_undegraded":"2024-09-05T13:12:03.072142+0000","last_fullsized":"2024-09-05T13:12:03.083665+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.085983+0000","last_change":"2024-09-05T13:12:03.085885+0000","last_active":"2024-09-05T13:12:03.085983+0000","last_peered":"2024-09-05T13:12:03.085983+0000","last_clean":"2024-09-05T13:12:03.085983+0000","last_became_active":"2024-09-05T13:12:03.085569+0000","last_became_peered":"2024-09-05T13:12:03.085569+0000","last_unstale":"2024-09-05T13:12:03.085983+0000","last_undegraded":"2024-09-05T13:12:03.085983+0000","last_fullsized":"2024-09-05T13:12:03.085983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:46:22.251600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":46,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.085360+0000","last_change":"2024-09-05T13:12:03.085535+0000","last_active":"2024-09-05T13:12:03.085360+0000","last_peered":"2024-09-05T13:12:03.085360+0000","last_clean":"2024-09-05T13:11:52.440198+0000","last_became_active":"2024-09-05T13:12:03.085276+0000","last_became_peered":"2024-09-05T13:12:03.085276+0000","last_unstale":"2024-09-05T13:12:03.085360+0000","last_undegraded":"2024-09-05T13:12:03.075750+0000","last_fullsized":"2024-09-05T13:12:03.085360+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.084841+0000","last_change":"2024-09-05T13:12:03.084744+0000","last_active":"2024-09-05T13:12:03.084841+0000","last_peered":"2024-09-05T13:12:03.084841+0000","last_clean":"2024-09-05T13:12:03.084841+0000","last_became_active":"2024-09-05T13:12:03.084462+0000","last_became_peered":"2024-09-05T13:12:03.084462+0000","last_unstale":"2024-09-05T13:12:03.084841+0000","last_undegraded":"2024-09-05T13:12:03.084841+0000","last_fullsized":"2024-09-05T13:12:03.084841+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:12:11.940605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":32,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.084086+0000","last_change":"2024-09-05T13:12:03.084261+0000","last_active":"2024-09-05T13:12:03.084086+0000","last_peered":"2024-09-05T13:12:03.084086+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:03.083997+0000","last_became_peered":"2024-09-05T13:12:03.083997+0000","last_unstale":"2024-09-05T13:12:03.084086+0000","last_undegraded":"2024-09-05T13:12:03.074451+0000","last_fullsized":"2024-09-05T13:12:03.084086+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.084636+0000","last_change":"2024-09-05T13:12:03.084538+0000","last_active":"2024-09-05T13:12:03.084636+0000","last_peered":"2024-09-05T13:12:03.084636+0000","last_clean":"2024-09-05T13:12:03.084636+0000","last_became_active":"2024-09-05T13:12:03.084046+0000","last_became_peered":"2024-09-05T13:12:03.084046+0000","last_unstale":"2024-09-05T13:12:03.084636+0000","last_undegraded":"2024-09-05T13:12:03.084636+0000","last_fullsized":"2024-09-05T13:12:03.084636+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:40:28.591527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":33,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.085634+0000","last_change":"2024-09-05T13:12:03.086027+0000","last_active":"2024-09-05T13:12:03.085634+0000","last_peered":"2024-09-05T13:12:03.085634+0000","last_clean":"2024-09-05T13:11:51.720508+0000","last_became_active":"2024-09-05T13:12:03.085548+0000","last_became_peered":"2024-09-05T13:12:03.085548+0000","last_unstale":"2024-09-05T13:12:03.085634+0000","last_undegraded":"2024-09-05T13:12:03.078105+0000","last_fullsized":"2024-09-05T13:12:03.085634+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":63,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.086379+0000","last_change":"2024-09-05T13:12:03.086259+0000","last_active":"2024-09-05T13:12:03.086379+0000","last_peered":"2024-09-05T13:12:03.086379+0000","last_clean":"2024-09-05T13:12:03.086379+0000","last_became_active":"2024-09-05T13:12:03.084878+0000","last_became_peered":"2024-09-05T13:12:03.084878+0000","last_unstale":"2024-09-05T13:12:03.086379+0000","last_undegraded":"2024-09-05T13:12:03.086379+0000","last_fullsized":"2024-09-05T13:12:03.086379+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:36.193136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":34,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.086935+0000","last_change":"2024-09-05T13:12:03.087118+0000","last_active":"2024-09-05T13:12:03.086935+0000","last_peered":"2024-09-05T13:12:03.086935+0000","last_clean":"2024-09-05T13:11:52.103995+0000","last_became_active":"2024-09-05T13:12:03.086762+0000","last_became_peered":"2024-09-05T13:12:03.086762+0000","last_unstale":"2024-09-05T13:12:03.086935+0000","last_undegraded":"2024-09-05T13:12:03.075362+0000","last_fullsized":"2024-09-05T13:12:03.086935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":37,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:03.084663+0000","last_change":"2024-09-05T13:12:03.088436+0000","last_active":"2024-09-05T13:12:03.084663+0000","last_peered":"2024-09-05T13:12:03.084663+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:03.084411+0000","last_became_peered":"2024-09-05T13:12:03.084411+0000","last_unstale":"2024-09-05T13:12:03.084663+0000","last_undegraded":"2024-09-05T13:12:03.076143+0000","last_fullsized":"2024-09-05T13:12:03.084663+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.085993+0000","last_change":"2024-09-05T13:12:03.085209+0000","last_active":"2024-09-05T13:12:03.085993+0000","last_peered":"2024-09-05T13:12:03.085993+0000","last_clean":"2024-09-05T13:12:03.085993+0000","last_became_active":"2024-09-05T13:12:03.084927+0000","last_became_peered":"2024-09-05T13:12:03.084927+0000","last_unstale":"2024-09-05T13:12:03.085993+0000","last_undegraded":"2024-09-05T13:12:03.085993+0000","last_fullsized":"2024-09-05T13:12:03.085993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:18:22.364531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":41,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.085714+0000","last_change":"2024-09-05T13:12:03.086004+0000","last_active":"2024-09-05T13:12:03.085714+0000","last_peered":"2024-09-05T13:12:03.085714+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:03.085630+0000","last_became_peered":"2024-09-05T13:12:03.085630+0000","last_unstale":"2024-09-05T13:12:03.085714+0000","last_undegraded":"2024-09-05T13:12:03.075626+0000","last_fullsized":"2024-09-05T13:12:03.085714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.084097+0000","last_change":"2024-09-05T13:12:03.083932+0000","last_active":"2024-09-05T13:12:03.084097+0000","last_peered":"2024-09-05T13:12:03.084097+0000","last_clean":"2024-09-05T13:12:03.084097+0000","last_became_active":"2024-09-05T13:12:03.083637+0000","last_became_peered":"2024-09-05T13:12:03.083637+0000","last_unstale":"2024-09-05T13:12:03.084097+0000","last_undegraded":"2024-09-05T13:12:03.084097+0000","last_fullsized":"2024-09-05T13:12:03.084097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:43.339163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":37,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.086004+0000","last_change":"2024-09-05T13:12:03.086181+0000","last_active":"2024-09-05T13:12:03.086004+0000","last_peered":"2024-09-05T13:12:03.086004+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:03.085915+0000","last_became_peered":"2024-09-05T13:12:03.085915+0000","last_unstale":"2024-09-05T13:12:03.086004+0000","last_undegraded":"2024-09-05T13:12:03.076498+0000","last_fullsized":"2024-09-05T13:12:03.086004+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.086353+0000","last_change":"2024-09-05T13:12:03.086248+0000","last_active":"2024-09-05T13:12:03.086353+0000","last_peered":"2024-09-05T13:12:03.086353+0000","last_clean":"2024-09-05T13:12:03.086353+0000","last_became_active":"2024-09-05T13:12:03.085407+0000","last_became_peered":"2024-09-05T13:12:03.085407+0000","last_unstale":"2024-09-05T13:12:03.086353+0000","last_undegraded":"2024-09-05T13:12:03.086353+0000","last_fullsized":"2024-09-05T13:12:03.086353+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:07:58.158386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-05T13:12:03.085983+0000","last_change":"2024-09-05T13:12:03.085893+0000","last_active":"2024-09-05T13:12:03.085983+0000","last_peered":"2024-09-05T13:12:03.085983+0000","last_clean":"2024-09-05T13:12:03.085983+0000","last_became_active":"2024-09-05T13:12:03.085584+0000","last_became_peered":"2024-09-05T13:12:03.085584+0000","last_unstale":"2024-09-05T13:12:03.085983+0000","last_undegraded":"2024-09-05T13:12:03.085983+0000","last_fullsized":"2024-09-05T13:12:03.085983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:07:47.618904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":45,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:03.085144+0000","last_change":"2024-09-05T13:12:03.085323+0000","last_active":"2024-09-05T13:12:03.085144+0000","last_peered":"2024-09-05T13:12:03.085144+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:03.085055+0000","last_became_peered":"2024-09-05T13:12:03.085055+0000","last_unstale":"2024-09-05T13:12:03.085144+0000","last_undegraded":"2024-09-05T13:12:03.075740+0000","last_fullsized":"2024-09-05T13:12:03.085144+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182403,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":266240,"data_stored":69417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738383,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":78111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:12:09.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:12:09.945 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:09.945 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":56,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":26,"num_osds":3,"num_up_osds":2,"osd_up_since":1725541922,"num_in_osds":2,"osd_in_since":1725541903,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56430592,"bytes_avail":193217097728,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-05T13:11:13.134280+0000","services":{}},"progress_events":{}} 2024-09-05T13:12:09.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '0162b639-bdbb-4bd7-a784-515f78b22316', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 56, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 26, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725541922, 'num_in_osds': 2, 'osd_in_since': 1725541903, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56430592, 'bytes_avail': 193217097728, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-05T13:11:13:133060+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-05T13:11:13.134280+0000', 'services': {}}, 'progress_events': {}} 2024-09-05T13:12:09.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-05T13:12:09.958 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-05T13:12:09.958 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:12:09.987 INFO:tasks.ceph.osd.1:Stopped 2024-09-05T13:12:09.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-05T13:12:10.322 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:12:10.321+0000 7f9a87bca640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-05T13:12:10.322 INFO:teuthology.orchestra.run.smithi086.stderr:osd.1 is already down. 2024-09-05T13:12:10.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-05T13:12:10.627 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:12:10.626+0000 7f9a87bca640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-05T13:12:11.098 INFO:teuthology.orchestra.run.smithi086.stderr:marked out osd.1. 2024-09-05T13:12:11.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-05T13:12:12.106 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.103+0000 7fc461b08640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-05T13:12:12.106 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.103+0000 7fc45cafe640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-05T13:12:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:marked osd lost in epoch 27 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.103+0000 7fc45db00640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.103+0000 7fc45d2ff640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.103+0000 7fc45bafc640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.103+0000 7fc461307640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.103+0000 7fc460305640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.103+0000 7fc45b2fb640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.104+0000 7fc45e301640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.104+0000 7fc45c2fd640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.104+0000 7fc45cafe640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.104+0000 7fc45f303640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-05T13:12:12.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.104+0000 7fc45aafa640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-05T13:12:12.109 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:12.104+0000 7fc45eb02640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-05T13:12:12.122 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-05T13:12:12.122 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-05T13:12:12.124 INFO:tasks.ceph.osd.2:Started 2024-09-05T13:12:12.124 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-05T13:12:12.227 INFO:teuthology.orchestra.run.smithi086.stderr:no valid command found; 10 closest matches: 2024-09-05T13:12:12.227 INFO:teuthology.orchestra.run.smithi086.stderr:0 2024-09-05T13:12:12.227 INFO:teuthology.orchestra.run.smithi086.stderr:1 2024-09-05T13:12:12.227 INFO:teuthology.orchestra.run.smithi086.stderr:2 2024-09-05T13:12:12.227 INFO:teuthology.orchestra.run.smithi086.stderr:abort 2024-09-05T13:12:12.227 INFO:teuthology.orchestra.run.smithi086.stderr:assert 2024-09-05T13:12:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:config diff 2024-09-05T13:12:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:config diff get 2024-09-05T13:12:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:config get 2024-09-05T13:12:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:config help [] 2024-09-05T13:12:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:config set ... 2024-09-05T13:12:12.228 INFO:teuthology.orchestra.run.smithi086.stderr:admin_socket: invalid command 2024-09-05T13:12:12.229 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-05T13:12:12.230 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-05T13:12:13.285 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-09-05T13:12:13.283+0000 7f5fc3e17640 -1 Falling back to public interface 2024-09-05T13:12:17.049 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-09-05T13:12:17.047+0000 7f5fc3e17640 -1 osd.2 17 log_to_monitors true 2024-09-05T13:12:17.107 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.103+0000 7fc45db00640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-05T13:12:17.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.103+0000 7fc45d2ff640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-05T13:12:17.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.103+0000 7fc461b08640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-05T13:12:17.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc45f303640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-05T13:12:17.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc461307640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-05T13:12:17.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc45e301640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-05T13:12:17.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc45bafc640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-05T13:12:17.108 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc45c2fd640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-05T13:12:17.109 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc45b2fb640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-05T13:12:17.109 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc460b06640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-05T13:12:17.109 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc45aafa640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-05T13:12:17.109 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc45c2fd640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-05T13:12:17.109 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc45eb02640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-05T13:12:17.109 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:17.104+0000 7fc460b06640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-05T13:12:17.231 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-05T13:12:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:no valid command found; 10 closest matches: 2024-09-05T13:12:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:0 2024-09-05T13:12:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:1 2024-09-05T13:12:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:2 2024-09-05T13:12:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:abort 2024-09-05T13:12:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:assert 2024-09-05T13:12:17.340 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs debug_inject_read_zeros 2024-09-05T13:12:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs files list 2024-09-05T13:12:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs stats 2024-09-05T13:12:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:bluestore allocator dump block 2024-09-05T13:12:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:bluestore allocator fragmentation block 2024-09-05T13:12:17.341 INFO:teuthology.orchestra.run.smithi086.stderr:admin_socket: invalid command 2024-09-05T13:12:17.342 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-05T13:12:17.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-05T13:12:17.848 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-09-05T13:12:17.845+0000 7f5fb7525640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T13:12:18.122 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.120+0000 7fc461b08640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-05T13:12:18.123 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.120+0000 7fc460305640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-05T13:12:18.123 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.120+0000 7fc45f303640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-05T13:12:18.123 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.120+0000 7fc45d2ff640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-05T13:12:18.123 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.120+0000 7fc45db00640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-05T13:12:18.124 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.121+0000 7fc45cafe640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-05T13:12:18.124 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.121+0000 7fc460305640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-05T13:12:18.125 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.122+0000 7fc460b06640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-05T13:12:18.125 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.122+0000 7fc45aafa640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-05T13:12:18.125 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.122+0000 7fc45b2fb640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-05T13:12:18.125 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.122+0000 7fc45bafc640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-05T13:12:18.125 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.122+0000 7fc45aafa640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-05T13:12:18.125 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.122+0000 7fc45a2f9640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-05T13:12:18.125 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:18.122+0000 7fc461307640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-05T13:12:19.128 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:19.125+0000 7fc45aafa640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-05T13:12:19.128 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:19.125+0000 7fc45f303640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-05T13:12:19.129 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:19.125+0000 7fc45cafe640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-05T13:12:19.129 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:19.125+0000 7fc460b06640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-05T13:12:19.129 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:19.126+0000 7fc45fb04640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-05T13:12:19.129 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:19.127+0000 7fc45d2ff640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-05T13:12:19.129 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:19.127+0000 7fc45a2f9640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-05T13:12:20.132 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:20.129+0000 7fc45cafe640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-05T13:12:20.132 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:20.130+0000 7fc460b06640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-05T13:12:20.132 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:20.130+0000 7fc45fb04640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-05T13:12:20.132 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:20.130+0000 7fc45e301640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-05T13:12:20.133 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:20.130+0000 7fc45f303640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-05T13:12:20.133 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:20.130+0000 7fc45eb02640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-05T13:12:20.133 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:20.130+0000 7fc461307640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-05T13:12:22.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-05T13:12:22.459 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-05T13:12:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-05T13:12:22.459 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:22.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-05T13:12:22.758 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:12:22.757+0000 7f9a87bca640 -1 mon.a@0(leader).osd e33 definitely_dead 0 2024-09-05T13:12:23.148 INFO:teuthology.orchestra.run.smithi086.stderr:marked in osd.2. 2024-09-05T13:12:23.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.2 to be up 2024-09-05T13:12:23.164 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T13:12:23.446 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:23.446 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":34,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:12:23.139660+0000","last_up_change":"2024-09-05T13:12:18.107914+0000","last_in_change":"2024-09-05T13:12:23.139660+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-05T13:11:28.452062+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-05T13:11:39.270087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":32,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6804","nonce":2531722421}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6805","nonce":2531722421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6807","nonce":2531722421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6806","nonce":2531722421}]},"public_addr":"172.21.15.86:6804/2531722421","cluster_addr":"172.21.15.86:6805/2531722421","heartbeat_back_addr":"172.21.15.86:6807/2531722421","heartbeat_front_addr":"172.21.15.86:6806/2531722421","state":["exists","up"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2999410110}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2999410110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2999410110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2999410110}]},"public_addr":"172.21.15.86:6800/2999410110","cluster_addr":"172.21.15.86:6801/2999410110","heartbeat_back_addr":"172.21.15.86:6803/2999410110","heartbeat_front_addr":"172.21.15.86:6802/2999410110","state":["exists"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":17,"up_from":31,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2895960470}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2895960470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2895960470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2895960470}]},"public_addr":"172.21.15.86:6800/2895960470","cluster_addr":"172.21.15.86:6801/2895960470","heartbeat_back_addr":"172.21.15.86:6803/2895960470","heartbeat_front_addr":"172.21.15.86:6802/2895960470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.643488+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-05T13:11:20.440515+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:21.020419+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:12:23.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.2 is up 2024-09-05T13:12:23.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-05T13:12:23.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-05T13:12:23.591 INFO:teuthology.orchestra.run.smithi086.stdout:133143986179 2024-09-05T13:12:23.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:12:23.593 INFO:teuthology.orchestra.run.smithi086.stdout:107374182407 2024-09-05T13:12:23.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:23.917 INFO:teuthology.orchestra.run.smithi086.stdout:107374182406 2024-09-05T13:12:23.923 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-09-05T13:12:23.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182406 for osd.0 2024-09-05T13:12:23.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 1 for osd.2 2024-09-05T13:12:24.931 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:24.937 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:12:25.162 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.160+0000 7fc461b08640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-05T13:12:25.162 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.160+0000 7fc45c2fd640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-05T13:12:25.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.161+0000 7fc45b2fb640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-05T13:12:25.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.161+0000 7fc45d2ff640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-05T13:12:25.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.161+0000 7fc45a2f9640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-05T13:12:25.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.161+0000 7fc45cafe640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-05T13:12:25.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.161+0000 7fc460b06640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-05T13:12:25.163 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.161+0000 7fc45bafc640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-05T13:12:25.164 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-09-05T13:12:25.161+0000 7fc45aafa640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-05T13:12:25.245 INFO:teuthology.orchestra.run.smithi086.stdout:107374182406 2024-09-05T13:12:25.258 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182406 for osd.0 2024-09-05T13:12:25.282 INFO:teuthology.orchestra.run.smithi086.stdout:133143986178 2024-09-05T13:12:25.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 133143986178 for osd.2 2024-09-05T13:12:26.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:26.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:12:26.567 INFO:teuthology.orchestra.run.smithi086.stdout:107374182407 2024-09-05T13:12:26.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182407 for osd.0 2024-09-05T13:12:26.581 DEBUG:teuthology.parallel:result is None 2024-09-05T13:12:26.604 INFO:teuthology.orchestra.run.smithi086.stdout:133143986179 2024-09-05T13:12:26.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 133143986179 for osd.2 2024-09-05T13:12:26.617 DEBUG:teuthology.parallel:result is None 2024-09-05T13:12:26.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-05T13:12:26.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:26.934 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:26.934 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:26.946 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-05T13:12:25.508122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55612,"kb_used_data":596,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688068,"statfs":{"total":193273528320,"available":193216581632,"internally_reserved":0,"allocated":610304,"data_stored":180312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":56286551},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.382944"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":40,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.131404+0000","last_change":"2024-09-05T13:12:23.149197+0000","last_active":"2024-09-05T13:12:23.149197+0000","last_peered":"2024-09-05T13:12:20.131404+0000","last_clean":"2024-09-05T13:12:03.089534+0000","last_became_active":"2024-09-05T13:12:11.105088+0000","last_became_peered":"2024-09-05T13:12:11.105088+0000","last_unstale":"2024-09-05T13:12:20.131404+0000","last_undegraded":"2024-09-05T13:12:20.131404+0000","last_fullsized":"2024-09-05T13:12:11.102548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":45,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.137011+0000","last_change":"2024-09-05T13:12:23.146221+0000","last_active":"2024-09-05T13:12:23.146221+0000","last_peered":"2024-09-05T13:12:20.137011+0000","last_clean":"2024-09-05T13:11:51.865448+0000","last_became_active":"2024-09-05T13:12:20.136715+0000","last_became_peered":"2024-09-05T13:12:20.136715+0000","last_unstale":"2024-09-05T13:12:20.137011+0000","last_undegraded":"2024-09-05T13:12:20.130160+0000","last_fullsized":"2024-09-05T13:12:20.137011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":60,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.138917+0000","last_change":"2024-09-05T13:12:23.146599+0000","last_active":"2024-09-05T13:12:23.146599+0000","last_peered":"2024-09-05T13:12:20.138917+0000","last_clean":"2024-09-05T13:11:52.391844+0000","last_became_active":"2024-09-05T13:12:20.138625+0000","last_became_peered":"2024-09-05T13:12:20.138625+0000","last_unstale":"2024-09-05T13:12:20.138917+0000","last_undegraded":"2024-09-05T13:12:20.130147+0000","last_fullsized":"2024-09-05T13:12:20.138917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.135752+0000","last_change":"2024-09-05T13:12:23.145574+0000","last_active":"2024-09-05T13:12:23.145574+0000","last_peered":"2024-09-05T13:12:20.135752+0000","last_clean":"2024-09-05T13:12:20.135752+0000","last_became_active":"2024-09-05T13:12:20.133612+0000","last_became_peered":"2024-09-05T13:12:20.133612+0000","last_unstale":"2024-09-05T13:12:20.135752+0000","last_undegraded":"2024-09-05T13:12:20.135752+0000","last_fullsized":"2024-09-05T13:12:20.135752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":57,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-05T13:12:23.148691+0000","last_change":"2024-09-05T13:12:23.148691+0000","last_active":"2024-09-05T13:12:23.148691+0000","last_peered":"2024-09-05T13:12:20.138495+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:20.138098+0000","last_became_peered":"2024-09-05T13:12:20.138098+0000","last_unstale":"2024-09-05T13:12:23.148691+0000","last_undegraded":"2024-09-05T13:12:23.148691+0000","last_fullsized":"2024-09-05T13:12:23.148691+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":52,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.130869+0000","last_change":"2024-09-05T13:12:23.149310+0000","last_active":"2024-09-05T13:12:23.149310+0000","last_peered":"2024-09-05T13:12:20.130869+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:11.104682+0000","last_became_peered":"2024-09-05T13:12:11.104682+0000","last_unstale":"2024-09-05T13:12:20.130869+0000","last_undegraded":"2024-09-05T13:12:11.101553+0000","last_fullsized":"2024-09-05T13:12:11.101440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":54,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.131595+0000","last_change":"2024-09-05T13:12:23.148403+0000","last_active":"2024-09-05T13:12:23.148403+0000","last_peered":"2024-09-05T13:12:20.131595+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:11.101802+0000","last_became_peered":"2024-09-05T13:12:11.101802+0000","last_unstale":"2024-09-05T13:12:20.131595+0000","last_undegraded":"2024-09-05T13:12:11.100492+0000","last_fullsized":"2024-09-05T13:12:11.100375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":54,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-05T13:12:23.146571+0000","last_change":"2024-09-05T13:12:23.146571+0000","last_active":"2024-09-05T13:12:23.146571+0000","last_peered":"2024-09-05T13:12:20.136381+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:20.135982+0000","last_became_peered":"2024-09-05T13:12:20.135982+0000","last_unstale":"2024-09-05T13:12:23.146571+0000","last_undegraded":"2024-09-05T13:12:23.146571+0000","last_fullsized":"2024-09-05T13:12:23.146571+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.130782+0000","last_change":"2024-09-05T13:12:23.147491+0000","last_active":"2024-09-05T13:12:23.147491+0000","last_peered":"2024-09-05T13:12:20.130782+0000","last_clean":"2024-09-05T13:11:44.973003+0000","last_became_active":"2024-09-05T13:12:11.101137+0000","last_became_peered":"2024-09-05T13:12:11.101137+0000","last_unstale":"2024-09-05T13:12:20.130782+0000","last_undegraded":"2024-09-05T13:12:20.130782+0000","last_fullsized":"2024-09-05T13:12:11.100001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":62,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.137793+0000","last_change":"2024-09-05T13:12:23.147515+0000","last_active":"2024-09-05T13:12:23.147515+0000","last_peered":"2024-09-05T13:12:20.137793+0000","last_clean":"2024-09-05T13:11:52.440198+0000","last_became_active":"2024-09-05T13:12:20.137517+0000","last_became_peered":"2024-09-05T13:12:20.137517+0000","last_unstale":"2024-09-05T13:12:20.137793+0000","last_undegraded":"2024-09-05T13:12:20.130398+0000","last_fullsized":"2024-09-05T13:12:20.137793+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.138270+0000","last_change":"2024-09-05T13:12:23.145592+0000","last_active":"2024-09-05T13:12:23.145592+0000","last_peered":"2024-09-05T13:12:20.138270+0000","last_clean":"2024-09-05T13:12:20.138270+0000","last_became_active":"2024-09-05T13:12:20.135683+0000","last_became_peered":"2024-09-05T13:12:20.135683+0000","last_unstale":"2024-09-05T13:12:20.138270+0000","last_undegraded":"2024-09-05T13:12:20.138270+0000","last_fullsized":"2024-09-05T13:12:20.138270+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":49,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.130576+0000","last_change":"2024-09-05T13:12:23.148609+0000","last_active":"2024-09-05T13:12:23.148609+0000","last_peered":"2024-09-05T13:12:20.130576+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:11.105013+0000","last_became_peered":"2024-09-05T13:12:11.105013+0000","last_unstale":"2024-09-05T13:12:20.130576+0000","last_undegraded":"2024-09-05T13:12:11.102043+0000","last_fullsized":"2024-09-05T13:12:11.101925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.130061+0000","last_change":"2024-09-05T13:12:23.146756+0000","last_active":"2024-09-05T13:12:23.146756+0000","last_peered":"2024-09-05T13:12:20.130061+0000","last_clean":"2024-09-05T13:12:03.089565+0000","last_became_active":"2024-09-05T13:12:11.101849+0000","last_became_peered":"2024-09-05T13:12:11.101849+0000","last_unstale":"2024-09-05T13:12:20.130061+0000","last_undegraded":"2024-09-05T13:12:11.100144+0000","last_fullsized":"2024-09-05T13:12:11.100062+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.134145+0000","last_change":"2024-09-05T13:12:23.146429+0000","last_active":"2024-09-05T13:12:23.146429+0000","last_peered":"2024-09-05T13:12:20.134145+0000","last_clean":"2024-09-05T13:11:51.720508+0000","last_became_active":"2024-09-05T13:12:20.133767+0000","last_became_peered":"2024-09-05T13:12:20.133767+0000","last_unstale":"2024-09-05T13:12:20.134145+0000","last_undegraded":"2024-09-05T13:12:20.130106+0000","last_fullsized":"2024-09-05T13:12:20.134145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":57,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.141561+0000","last_change":"2024-09-05T13:12:23.146354+0000","last_active":"2024-09-05T13:12:23.146354+0000","last_peered":"2024-09-05T13:12:20.141561+0000","last_clean":"2024-09-05T13:12:20.141561+0000","last_became_active":"2024-09-05T13:12:20.137421+0000","last_became_peered":"2024-09-05T13:12:20.137421+0000","last_unstale":"2024-09-05T13:12:20.141561+0000","last_undegraded":"2024-09-05T13:12:20.141561+0000","last_fullsized":"2024-09-05T13:12:20.141561+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.136257+0000","last_change":"2024-09-05T13:12:23.146347+0000","last_active":"2024-09-05T13:12:23.146347+0000","last_peered":"2024-09-05T13:12:20.136257+0000","last_clean":"2024-09-05T13:11:52.103995+0000","last_became_active":"2024-09-05T13:12:20.135217+0000","last_became_peered":"2024-09-05T13:12:20.135217+0000","last_unstale":"2024-09-05T13:12:20.136257+0000","last_undegraded":"2024-09-05T13:12:20.129774+0000","last_fullsized":"2024-09-05T13:12:20.136257+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":49,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.131097+0000","last_change":"2024-09-05T13:12:23.148665+0000","last_active":"2024-09-05T13:12:23.148665+0000","last_peered":"2024-09-05T13:12:20.131097+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:11.105353+0000","last_became_peered":"2024-09-05T13:12:11.105353+0000","last_unstale":"2024-09-05T13:12:20.131097+0000","last_undegraded":"2024-09-05T13:12:11.102654+0000","last_fullsized":"2024-09-05T13:12:11.102546+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.131135+0000","last_change":"2024-09-05T13:12:23.147541+0000","last_active":"2024-09-05T13:12:23.147541+0000","last_peered":"2024-09-05T13:12:20.131135+0000","last_clean":"2024-09-05T13:11:44.973098+0000","last_became_active":"2024-09-05T13:12:11.104755+0000","last_became_peered":"2024-09-05T13:12:11.104755+0000","last_unstale":"2024-09-05T13:12:20.131135+0000","last_undegraded":"2024-09-05T13:12:20.131135+0000","last_fullsized":"2024-09-05T13:12:11.102130+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":54,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.131434+0000","last_change":"2024-09-05T13:12:23.146978+0000","last_active":"2024-09-05T13:12:23.146978+0000","last_peered":"2024-09-05T13:12:20.131434+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:11.105612+0000","last_became_peered":"2024-09-05T13:12:11.105612+0000","last_unstale":"2024-09-05T13:12:20.131434+0000","last_undegraded":"2024-09-05T13:12:11.102621+0000","last_fullsized":"2024-09-05T13:12:11.102539+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.141905+0000","last_change":"2024-09-05T13:12:23.145841+0000","last_active":"2024-09-05T13:12:23.145841+0000","last_peered":"2024-09-05T13:12:20.141905+0000","last_clean":"2024-09-05T13:12:20.141905+0000","last_became_active":"2024-09-05T13:12:20.138397+0000","last_became_peered":"2024-09-05T13:12:20.138397+0000","last_unstale":"2024-09-05T13:12:20.141905+0000","last_undegraded":"2024-09-05T13:12:20.141905+0000","last_fullsized":"2024-09-05T13:12:20.141905+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":50,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.131288+0000","last_change":"2024-09-05T13:12:23.148528+0000","last_active":"2024-09-05T13:12:23.148528+0000","last_peered":"2024-09-05T13:12:20.131288+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:11.105119+0000","last_became_peered":"2024-09-05T13:12:11.105119+0000","last_unstale":"2024-09-05T13:12:20.131288+0000","last_undegraded":"2024-09-05T13:12:11.102241+0000","last_fullsized":"2024-09-05T13:12:11.102133+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.130839+0000","last_change":"2024-09-05T13:12:23.146505+0000","last_active":"2024-09-05T13:12:23.146505+0000","last_peered":"2024-09-05T13:12:20.130839+0000","last_clean":"2024-09-05T13:11:44.973212+0000","last_became_active":"2024-09-05T13:12:11.104737+0000","last_became_peered":"2024-09-05T13:12:11.104737+0000","last_unstale":"2024-09-05T13:12:20.130839+0000","last_undegraded":"2024-09-05T13:12:20.130839+0000","last_fullsized":"2024-09-05T13:12:11.102113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-05T13:12:20.138305+0000","last_change":"2024-09-05T13:12:23.147381+0000","last_active":"2024-09-05T13:12:23.147381+0000","last_peered":"2024-09-05T13:12:20.138305+0000","last_clean":"2024-09-05T13:12:20.138305+0000","last_became_active":"2024-09-05T13:12:20.135157+0000","last_became_peered":"2024-09-05T13:12:20.135157+0000","last_unstale":"2024-09-05T13:12:20.138305+0000","last_undegraded":"2024-09-05T13:12:20.138305+0000","last_fullsized":"2024-09-05T13:12:20.138305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":58,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-05T13:12:20.130919+0000","last_change":"2024-09-05T13:12:23.148345+0000","last_active":"2024-09-05T13:12:23.148345+0000","last_peered":"2024-09-05T13:12:20.130919+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:11.104743+0000","last_became_peered":"2024-09-05T13:12:11.104743+0000","last_unstale":"2024-09-05T13:12:20.130919+0000","last_undegraded":"2024-09-05T13:12:11.101928+0000","last_fullsized":"2024-09-05T13:12:11.101790+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":278528,"data_stored":86060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28176045},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182407,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":331776,"data_stored":94252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:12:26.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:12:27.298 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:27.298 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1725541938,"num_in_osds":2,"osd_in_since":1725541943,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56946688,"bytes_avail":193216581632,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-05T13:12:23.507777+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:12:27.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '0162b639-bdbb-4bd7-a784-515f78b22316', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 74, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725541938, 'num_in_osds': 2, 'osd_in_since': 1725541943, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56946688, 'bytes_avail': 193216581632, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-05T13:11:13:133060+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-05T13:12:23.507777+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-05T13:12:28.171 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:12:28.170+0000 7f9a8a3cf640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-05T13:12:30.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:30.604 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:30.605 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:30.616 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-05T13:12:29.509019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55660,"kb_used_data":636,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688020,"statfs":{"total":193273528320,"available":193216532480,"internally_reserved":0,"allocated":651264,"data_stored":194911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":56286551},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-6,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.365992"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":51,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162400+0000","last_change":"2024-09-05T13:12:24.168600+0000","last_active":"2024-09-05T13:12:25.162400+0000","last_peered":"2024-09-05T13:12:25.162400+0000","last_clean":"2024-09-05T13:12:25.162400+0000","last_became_active":"2024-09-05T13:12:24.168356+0000","last_became_peered":"2024-09-05T13:12:24.168356+0000","last_unstale":"2024-09-05T13:12:25.162400+0000","last_undegraded":"2024-09-05T13:12:25.162400+0000","last_fullsized":"2024-09-05T13:12:25.162400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:44.376826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":49,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-05T13:12:25.171316+0000","last_change":"2024-09-05T13:12:25.171469+0000","last_active":"2024-09-05T13:12:25.171316+0000","last_peered":"2024-09-05T13:12:25.171316+0000","last_clean":"2024-09-05T13:11:51.865448+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:12:25.171316+0000","last_undegraded":"2024-09-05T13:12:25.162431+0000","last_fullsized":"2024-09-05T13:12:25.171316+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":65,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-05T13:12:25.171056+0000","last_change":"2024-09-05T13:12:27.115370+0000","last_active":"2024-09-05T13:12:25.171056+0000","last_peered":"2024-09-05T13:12:25.171056+0000","last_clean":"2024-09-05T13:11:52.391844+0000","last_became_active":"2024-09-05T13:12:25.166237+0000","last_became_peered":"2024-09-05T13:12:25.166237+0000","last_unstale":"2024-09-05T13:12:25.171056+0000","last_undegraded":"2024-09-05T13:12:25.171056+0000","last_fullsized":"2024-09-05T13:12:25.171056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.167345+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:12:25.167345+0000","last_peered":"2024-09-05T13:12:25.167345+0000","last_clean":"2024-09-05T13:12:25.167345+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:12:25.167345+0000","last_undegraded":"2024-09-05T13:12:25.167345+0000","last_fullsized":"2024-09-05T13:12:25.167345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":65,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.161503+0000","last_change":"2024-09-05T13:12:24.165681+0000","last_active":"2024-09-05T13:12:25.161503+0000","last_peered":"2024-09-05T13:12:25.161503+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:12:25.161503+0000","last_undegraded":"2024-09-05T13:12:24.156268+0000","last_fullsized":"2024-09-05T13:12:25.161503+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162286+0000","last_change":"2024-09-05T13:12:24.165709+0000","last_active":"2024-09-05T13:12:25.162286+0000","last_peered":"2024-09-05T13:12:25.162286+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:24.165469+0000","last_became_peered":"2024-09-05T13:12:24.165469+0000","last_unstale":"2024-09-05T13:12:25.162286+0000","last_undegraded":"2024-09-05T13:12:24.157014+0000","last_fullsized":"2024-09-05T13:12:25.162286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162114+0000","last_change":"2024-09-05T13:12:24.165854+0000","last_active":"2024-09-05T13:12:25.162114+0000","last_peered":"2024-09-05T13:12:25.162114+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:24.165604+0000","last_became_peered":"2024-09-05T13:12:24.165604+0000","last_unstale":"2024-09-05T13:12:25.162114+0000","last_undegraded":"2024-09-05T13:12:24.157554+0000","last_fullsized":"2024-09-05T13:12:25.162114+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.161105+0000","last_change":"2024-09-05T13:12:24.163654+0000","last_active":"2024-09-05T13:12:25.161105+0000","last_peered":"2024-09-05T13:12:25.161105+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:12:25.161105+0000","last_undegraded":"2024-09-05T13:12:24.156202+0000","last_fullsized":"2024-09-05T13:12:25.161105+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.160861+0000","last_change":"2024-09-05T13:12:24.163255+0000","last_active":"2024-09-05T13:12:25.160861+0000","last_peered":"2024-09-05T13:12:25.160861+0000","last_clean":"2024-09-05T13:12:25.160861+0000","last_became_active":"2024-09-05T13:12:24.162958+0000","last_became_peered":"2024-09-05T13:12:24.162958+0000","last_unstale":"2024-09-05T13:12:25.160861+0000","last_undegraded":"2024-09-05T13:12:25.160861+0000","last_fullsized":"2024-09-05T13:12:25.160861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:58.405279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":67,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-05T13:12:25.171963+0000","last_change":"2024-09-05T13:12:27.115290+0000","last_active":"2024-09-05T13:12:25.171963+0000","last_peered":"2024-09-05T13:12:25.171963+0000","last_clean":"2024-09-05T13:11:52.440198+0000","last_became_active":"2024-09-05T13:12:25.167319+0000","last_became_peered":"2024-09-05T13:12:25.167319+0000","last_unstale":"2024-09-05T13:12:25.171963+0000","last_undegraded":"2024-09-05T13:12:25.171963+0000","last_fullsized":"2024-09-05T13:12:25.171963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.165344+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:12:25.165344+0000","last_peered":"2024-09-05T13:12:25.165344+0000","last_clean":"2024-09-05T13:12:25.165344+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:12:25.165344+0000","last_undegraded":"2024-09-05T13:12:25.165344+0000","last_fullsized":"2024-09-05T13:12:25.165344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":57,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162350+0000","last_change":"2024-09-05T13:12:24.165760+0000","last_active":"2024-09-05T13:12:25.162350+0000","last_peered":"2024-09-05T13:12:25.162350+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:24.165511+0000","last_became_peered":"2024-09-05T13:12:24.165511+0000","last_unstale":"2024-09-05T13:12:25.162350+0000","last_undegraded":"2024-09-05T13:12:24.157434+0000","last_fullsized":"2024-09-05T13:12:25.162350+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162067+0000","last_change":"2024-09-05T13:12:24.164231+0000","last_active":"2024-09-05T13:12:25.162067+0000","last_peered":"2024-09-05T13:12:25.162067+0000","last_clean":"2024-09-05T13:12:03.089565+0000","last_became_active":"2024-09-05T13:12:24.163048+0000","last_became_peered":"2024-09-05T13:12:24.163048+0000","last_unstale":"2024-09-05T13:12:25.162067+0000","last_undegraded":"2024-09-05T13:12:24.156585+0000","last_fullsized":"2024-09-05T13:12:25.162067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":53,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-05T13:12:25.170231+0000","last_change":"2024-09-05T13:12:25.170512+0000","last_active":"2024-09-05T13:12:25.170231+0000","last_peered":"2024-09-05T13:12:25.170231+0000","last_clean":"2024-09-05T13:11:51.720508+0000","last_became_active":"2024-09-05T13:12:25.165724+0000","last_became_peered":"2024-09-05T13:12:25.165724+0000","last_unstale":"2024-09-05T13:12:25.170231+0000","last_undegraded":"2024-09-05T13:12:25.161640+0000","last_fullsized":"2024-09-05T13:12:25.170231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.169048+0000","last_change":"2024-09-05T13:12:25.167242+0000","last_active":"2024-09-05T13:12:25.169048+0000","last_peered":"2024-09-05T13:12:25.169048+0000","last_clean":"2024-09-05T13:12:25.169048+0000","last_became_active":"2024-09-05T13:12:25.165964+0000","last_became_peered":"2024-09-05T13:12:25.165964+0000","last_unstale":"2024-09-05T13:12:25.169048+0000","last_undegraded":"2024-09-05T13:12:25.169048+0000","last_fullsized":"2024-09-05T13:12:25.169048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:03.155260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":55,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-05T13:12:25.169586+0000","last_change":"2024-09-05T13:12:25.169872+0000","last_active":"2024-09-05T13:12:25.169586+0000","last_peered":"2024-09-05T13:12:25.169586+0000","last_clean":"2024-09-05T13:11:52.103995+0000","last_became_active":"2024-09-05T13:12:25.164969+0000","last_became_peered":"2024-09-05T13:12:25.164969+0000","last_unstale":"2024-09-05T13:12:25.169586+0000","last_undegraded":"2024-09-05T13:12:25.161065+0000","last_fullsized":"2024-09-05T13:12:25.169586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":57,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162201+0000","last_change":"2024-09-05T13:12:24.168530+0000","last_active":"2024-09-05T13:12:25.162201+0000","last_peered":"2024-09-05T13:12:25.162201+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:24.168300+0000","last_became_peered":"2024-09-05T13:12:24.168300+0000","last_unstale":"2024-09-05T13:12:25.162201+0000","last_undegraded":"2024-09-05T13:12:24.158160+0000","last_fullsized":"2024-09-05T13:12:25.162201+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162822+0000","last_change":"2024-09-05T13:12:24.163476+0000","last_active":"2024-09-05T13:12:25.162822+0000","last_peered":"2024-09-05T13:12:25.162822+0000","last_clean":"2024-09-05T13:12:25.162822+0000","last_became_active":"2024-09-05T13:12:24.163169+0000","last_became_peered":"2024-09-05T13:12:24.163169+0000","last_unstale":"2024-09-05T13:12:25.162822+0000","last_undegraded":"2024-09-05T13:12:25.162822+0000","last_fullsized":"2024-09-05T13:12:25.162822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:05:50.037488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162647+0000","last_change":"2024-09-05T13:12:24.168576+0000","last_active":"2024-09-05T13:12:25.162647+0000","last_peered":"2024-09-05T13:12:25.162647+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:24.168347+0000","last_became_peered":"2024-09-05T13:12:24.168347+0000","last_unstale":"2024-09-05T13:12:25.162647+0000","last_undegraded":"2024-09-05T13:12:24.157388+0000","last_fullsized":"2024-09-05T13:12:25.162647+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.166730+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:12:25.166730+0000","last_peered":"2024-09-05T13:12:25.166730+0000","last_clean":"2024-09-05T13:12:25.166730+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:12:25.166730+0000","last_undegraded":"2024-09-05T13:12:25.166730+0000","last_fullsized":"2024-09-05T13:12:25.166730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":58,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.161899+0000","last_change":"2024-09-05T13:12:24.165983+0000","last_active":"2024-09-05T13:12:25.161899+0000","last_peered":"2024-09-05T13:12:25.161899+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:24.165544+0000","last_became_peered":"2024-09-05T13:12:24.165544+0000","last_unstale":"2024-09-05T13:12:25.161899+0000","last_undegraded":"2024-09-05T13:12:24.157734+0000","last_fullsized":"2024-09-05T13:12:25.161899+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162476+0000","last_change":"2024-09-05T13:12:24.165771+0000","last_active":"2024-09-05T13:12:25.162476+0000","last_peered":"2024-09-05T13:12:25.162476+0000","last_clean":"2024-09-05T13:12:25.162476+0000","last_became_active":"2024-09-05T13:12:24.165476+0000","last_became_peered":"2024-09-05T13:12:24.165476+0000","last_unstale":"2024-09-05T13:12:25.162476+0000","last_undegraded":"2024-09-05T13:12:25.162476+0000","last_fullsized":"2024-09-05T13:12:25.162476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:24.468878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.168816+0000","last_change":"2024-09-05T13:12:25.166728+0000","last_active":"2024-09-05T13:12:25.168816+0000","last_peered":"2024-09-05T13:12:25.168816+0000","last_clean":"2024-09-05T13:12:25.168816+0000","last_became_active":"2024-09-05T13:12:25.166434+0000","last_became_peered":"2024-09-05T13:12:25.166434+0000","last_unstale":"2024-09-05T13:12:25.168816+0000","last_undegraded":"2024-09-05T13:12:25.168816+0000","last_fullsized":"2024-09-05T13:12:25.168816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:26:25.045325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":67,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:25.162543+0000","last_change":"2024-09-05T13:12:24.164240+0000","last_active":"2024-09-05T13:12:25.162543+0000","last_peered":"2024-09-05T13:12:25.162543+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:24.163127+0000","last_became_peered":"2024-09-05T13:12:24.163127+0000","last_unstale":"2024-09-05T13:12:25.162543+0000","last_undegraded":"2024-09-05T13:12:24.155686+0000","last_fullsized":"2024-09-05T13:12:25.162543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986180,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":296,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":303104,"data_stored":95031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":28176045},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182408,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":348160,"data_stored":99880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:12:30.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:12:30.965 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:30.965 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":77,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1725541938,"num_in_osds":2,"osd_in_since":1725541943,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56995840,"bytes_avail":193216532480,"bytes_total":193273528320,"degraded_objects":50,"degraded_total":60,"degraded_ratio":0.83333333333333348,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-05T13:12:23.507777+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:12:30.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '0162b639-bdbb-4bd7-a784-515f78b22316', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded', 'count': 12}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 77, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725541938, 'num_in_osds': 2, 'osd_in_since': 1725541943, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56995840, 'bytes_avail': 193216532480, 'bytes_total': 193273528320, 'degraded_objects': 50, 'degraded_total': 60, 'degraded_ratio': 0.8333333333333335, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-05T13:11:13:133060+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-05T13:12:23.507777+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-05T13:12:30.978 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-05T13:12:30.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-05T13:12:30.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-05T13:12:31.110 INFO:teuthology.orchestra.run.smithi086.stdout:133143986181 2024-09-05T13:12:31.111 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:12:31.112 INFO:teuthology.orchestra.run.smithi086.stdout:107374182410 2024-09-05T13:12:31.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:31.443 INFO:teuthology.orchestra.run.smithi086.stdout:107374182408 2024-09-05T13:12:31.444 INFO:teuthology.orchestra.run.smithi086.stdout:133143986180 2024-09-05T13:12:31.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182410 got 107374182408 for osd.0 2024-09-05T13:12:31.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986181 got 133143986180 for osd.2 2024-09-05T13:12:32.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:32.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:12:32.763 INFO:teuthology.orchestra.run.smithi086.stdout:107374182410 2024-09-05T13:12:32.764 INFO:teuthology.orchestra.run.smithi086.stdout:133143986181 2024-09-05T13:12:32.778 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986181 got 133143986181 for osd.2 2024-09-05T13:12:32.778 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182410 got 107374182410 for osd.0 2024-09-05T13:12:32.778 DEBUG:teuthology.parallel:result is None 2024-09-05T13:12:32.778 DEBUG:teuthology.parallel:result is None 2024-09-05T13:12:32.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:12:33.122 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:33.122 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":79,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1725541938,"num_in_osds":2,"osd_in_since":1725541943,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56995840,"bytes_avail":193216532480,"bytes_total":193273528320,"degraded_objects":50,"degraded_total":60,"degraded_ratio":0.83333333333333348,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-05T13:12:23.507777+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:12:33.134 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '0162b639-bdbb-4bd7-a784-515f78b22316', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded', 'count': 12}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 79, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1725541938, 'num_in_osds': 2, 'osd_in_since': 1725541943, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56995840, 'bytes_avail': 193216532480, 'bytes_total': 193273528320, 'degraded_objects': 50, 'degraded_total': 60, 'degraded_ratio': 0.8333333333333335, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-05T13:11:13:133060+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-05T13:12:23.507777+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-05T13:12:33.134 INFO:tasks.rep_lost_unfound_delete:there are 27 unfound objects 2024-09-05T13:12:33.135 DEBUG:teuthology.orchestra.run.smithi086:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfounddel_pool -t 20 bench 240 write' 2024-09-05T13:12:33.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:hints = 1 2024-09-05T13:12:33.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-05T13:12:33.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Object prefix: benchmark_data_smithi086_44004 2024-09-05T13:12:33.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:12:33.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 0 0 0 0 0 0 - 0 2024-09-05T13:12:34.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 1 20 3221 3201 12.5032 12.5039 0.00430462 0.0062246 2024-09-05T13:12:35.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 2 20 6458 6438 12.5734 12.6445 0.00782699 0.00620122 2024-09-05T13:12:36.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 3 20 9529 9509 12.3806 11.9961 0.00755122 0.0062997 2024-09-05T13:12:37.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 4 20 12510 12490 12.1961 11.6445 0.00630449 0.00639828 2024-09-05T13:12:38.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 5 20 15382 15362 12.0005 11.2188 0.00484666 0.00650143 2024-09-05T13:12:39.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 6 19 18117 18098 11.7815 10.6875 0.00987235 0.00662368 2024-09-05T13:12:40.185 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 7 20 20756 20736 11.5704 10.3047 0.00998114 0.0067449 2024-09-05T13:12:41.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 8 20 23217 23197 11.3254 9.61328 0.0101516 0.00689197 2024-09-05T13:12:42.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 9 20 25594 25574 11.0987 9.28516 0.00554257 0.00703254 2024-09-05T13:12:43.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:43.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 10 20 27861 27841 10.8742 8.85547 0.0089126 0.00717679 2024-09-05T13:12:43.672 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:43.673 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:43.695 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-05T13:12:43.512519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97957095,"num_objects":23942,"num_object_clones":0,"num_object_copies":47884,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":23942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23969,"num_write_kb":95696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23970,"ondisk_log_size":23970,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":326000,"kb_used_data":177996,"kb_used_omap":8,"kb_used_meta":147959,"kb_avail":188417680,"statfs":{"total":193273528320,"available":192939704320,"internally_reserved":0,"allocated":182267904,"data_stored":181811551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":151510359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,5,9,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":97943552,"num_objects":23912,"num_object_clones":0,"num_object_copies":47824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23912,"num_write_kb":95648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002940"},"pg_stats":[{"pgid":"2.f","version":"36'1409","reported_seq":1459,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:41.078939+0000","last_change":"2024-09-05T13:12:24.168600+0000","last_active":"2024-09-05T13:12:41.078939+0000","last_peered":"2024-09-05T13:12:41.078939+0000","last_clean":"2024-09-05T13:12:41.078939+0000","last_became_active":"2024-09-05T13:12:24.168356+0000","last_became_peered":"2024-09-05T13:12:24.168356+0000","last_unstale":"2024-09-05T13:12:41.078939+0000","last_undegraded":"2024-09-05T13:12:41.078939+0000","last_fullsized":"2024-09-05T13:12:41.078939+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:44.376826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5771264,"num_objects":1409,"num_object_clones":0,"num_object_copies":2818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1409,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'1556","reported_seq":1601,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-05T13:12:42.105555+0000","last_change":"2024-09-05T13:12:33.192792+0000","last_active":"2024-09-05T13:12:42.105555+0000","last_peered":"2024-09-05T13:12:42.105555+0000","last_clean":"2024-09-05T13:11:51.865448+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:12:42.105555+0000","last_undegraded":"2024-09-05T13:12:25.171469+0000","last_fullsized":"2024-09-05T13:12:42.105555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1556,"log_dups_size":0,"ondisk_log_size":1556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6357958,"num_objects":1554,"num_object_clones":0,"num_object_copies":3108,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1556,"num_write_kb":6212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'1638","reported_seq":1700,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-05T13:12:42.110013+0000","last_change":"2024-09-05T13:12:27.115370+0000","last_active":"2024-09-05T13:12:42.110013+0000","last_peered":"2024-09-05T13:12:42.110013+0000","last_clean":"2024-09-05T13:11:52.391844+0000","last_became_active":"2024-09-05T13:12:25.166237+0000","last_became_peered":"2024-09-05T13:12:25.166237+0000","last_unstale":"2024-09-05T13:12:42.110013+0000","last_undegraded":"2024-09-05T13:12:25.171056+0000","last_fullsized":"2024-09-05T13:12:42.110013+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1638,"log_dups_size":0,"ondisk_log_size":1638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6697443,"num_objects":1636,"num_object_clones":0,"num_object_copies":3272,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1638,"num_write_kb":6542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1636}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'1561","reported_seq":1611,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:42.119413+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:12:42.119413+0000","last_peered":"2024-09-05T13:12:42.119413+0000","last_clean":"2024-09-05T13:12:42.119413+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:12:42.119413+0000","last_undegraded":"2024-09-05T13:12:42.119413+0000","last_fullsized":"2024-09-05T13:12:42.119413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1561,"log_dups_size":0,"ondisk_log_size":1561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6393856,"num_objects":1561,"num_object_clones":0,"num_object_copies":3122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1561,"num_write_kb":6244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'1468","reported_seq":1528,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:41.069268+0000","last_change":"2024-09-05T13:12:24.165681+0000","last_active":"2024-09-05T13:12:41.069268+0000","last_peered":"2024-09-05T13:12:41.069268+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:12:41.069268+0000","last_undegraded":"2024-09-05T13:12:24.156268+0000","last_fullsized":"2024-09-05T13:12:41.069268+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1468,"log_dups_size":0,"ondisk_log_size":1468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5993414,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1468,"num_write_kb":5856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'1430","reported_seq":1487,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:41.084911+0000","last_change":"2024-09-05T13:12:24.165709+0000","last_active":"2024-09-05T13:12:41.084911+0000","last_peered":"2024-09-05T13:12:41.084911+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:24.165469+0000","last_became_peered":"2024-09-05T13:12:24.165469+0000","last_unstale":"2024-09-05T13:12:41.084911+0000","last_undegraded":"2024-09-05T13:12:24.157014+0000","last_fullsized":"2024-09-05T13:12:41.084911+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1430,"log_dups_size":0,"ondisk_log_size":1430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5845958,"num_objects":1429,"num_object_clones":0,"num_object_copies":2858,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1430,"num_write_kb":5711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1429}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'1397","reported_seq":1455,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:41.081626+0000","last_change":"2024-09-05T13:12:24.165854+0000","last_active":"2024-09-05T13:12:41.081626+0000","last_peered":"2024-09-05T13:12:41.081626+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:24.165604+0000","last_became_peered":"2024-09-05T13:12:24.165604+0000","last_unstale":"2024-09-05T13:12:41.081626+0000","last_undegraded":"2024-09-05T13:12:24.157554+0000","last_fullsized":"2024-09-05T13:12:41.081626+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1397,"log_dups_size":0,"ondisk_log_size":1397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5706694,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1397,"num_write_kb":5576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'1447","reported_seq":1508,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:41.080577+0000","last_change":"2024-09-05T13:12:24.163654+0000","last_active":"2024-09-05T13:12:41.080577+0000","last_peered":"2024-09-05T13:12:41.080577+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:12:41.080577+0000","last_undegraded":"2024-09-05T13:12:24.156202+0000","last_fullsized":"2024-09-05T13:12:41.080577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5923299,"num_objects":1447,"num_object_clones":0,"num_object_copies":2894,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1447,"num_write_kb":5785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1447}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.160861+0000","last_change":"2024-09-05T13:12:24.163255+0000","last_active":"2024-09-05T13:12:25.160861+0000","last_peered":"2024-09-05T13:12:25.160861+0000","last_clean":"2024-09-05T13:12:25.160861+0000","last_became_active":"2024-09-05T13:12:24.162958+0000","last_became_peered":"2024-09-05T13:12:24.162958+0000","last_unstale":"2024-09-05T13:12:25.160861+0000","last_undegraded":"2024-09-05T13:12:25.160861+0000","last_fullsized":"2024-09-05T13:12:25.160861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:58.405279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'1620","reported_seq":1681,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-05T13:12:42.121078+0000","last_change":"2024-09-05T13:12:27.115290+0000","last_active":"2024-09-05T13:12:42.121078+0000","last_peered":"2024-09-05T13:12:42.121078+0000","last_clean":"2024-09-05T13:11:52.440198+0000","last_became_active":"2024-09-05T13:12:25.167319+0000","last_became_peered":"2024-09-05T13:12:25.167319+0000","last_unstale":"2024-09-05T13:12:42.121078+0000","last_undegraded":"2024-09-05T13:12:25.171963+0000","last_fullsized":"2024-09-05T13:12:42.121078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1620,"log_dups_size":0,"ondisk_log_size":1620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6616006,"num_objects":1617,"num_object_clones":0,"num_object_copies":3234,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1620,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1617}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.165344+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:12:25.165344+0000","last_peered":"2024-09-05T13:12:25.165344+0000","last_clean":"2024-09-05T13:12:25.165344+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:12:25.165344+0000","last_undegraded":"2024-09-05T13:12:25.165344+0000","last_fullsized":"2024-09-05T13:12:25.165344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'1415","reported_seq":1466,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:41.075244+0000","last_change":"2024-09-05T13:12:24.165760+0000","last_active":"2024-09-05T13:12:41.075244+0000","last_peered":"2024-09-05T13:12:41.075244+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:24.165511+0000","last_became_peered":"2024-09-05T13:12:24.165511+0000","last_unstale":"2024-09-05T13:12:41.075244+0000","last_undegraded":"2024-09-05T13:12:24.157434+0000","last_fullsized":"2024-09-05T13:12:41.075244+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1415,"log_dups_size":0,"ondisk_log_size":1415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5772230,"num_objects":1411,"num_object_clones":0,"num_object_copies":2822,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1411}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162067+0000","last_change":"2024-09-05T13:12:24.164231+0000","last_active":"2024-09-05T13:12:25.162067+0000","last_peered":"2024-09-05T13:12:25.162067+0000","last_clean":"2024-09-05T13:12:03.089565+0000","last_became_active":"2024-09-05T13:12:24.163048+0000","last_became_peered":"2024-09-05T13:12:24.163048+0000","last_unstale":"2024-09-05T13:12:25.162067+0000","last_undegraded":"2024-09-05T13:12:24.156585+0000","last_fullsized":"2024-09-05T13:12:25.162067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'1633","reported_seq":1682,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-05T13:12:42.117105+0000","last_change":"2024-09-05T13:12:33.192873+0000","last_active":"2024-09-05T13:12:42.117105+0000","last_peered":"2024-09-05T13:12:42.117105+0000","last_clean":"2024-09-05T13:11:51.720508+0000","last_became_active":"2024-09-05T13:12:25.165724+0000","last_became_peered":"2024-09-05T13:12:25.165724+0000","last_unstale":"2024-09-05T13:12:42.117105+0000","last_undegraded":"2024-09-05T13:12:25.170512+0000","last_fullsized":"2024-09-05T13:12:42.117105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1633,"log_dups_size":0,"ondisk_log_size":1633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6677446,"num_objects":1632,"num_object_clones":0,"num_object_copies":3264,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1633,"num_write_kb":6523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1632}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.169048+0000","last_change":"2024-09-05T13:12:25.167242+0000","last_active":"2024-09-05T13:12:25.169048+0000","last_peered":"2024-09-05T13:12:25.169048+0000","last_clean":"2024-09-05T13:12:25.169048+0000","last_became_active":"2024-09-05T13:12:25.165964+0000","last_became_peered":"2024-09-05T13:12:25.165964+0000","last_unstale":"2024-09-05T13:12:25.169048+0000","last_undegraded":"2024-09-05T13:12:25.169048+0000","last_fullsized":"2024-09-05T13:12:25.169048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:03.155260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'1565","reported_seq":1616,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-05T13:12:42.120216+0000","last_change":"2024-09-05T13:12:33.192988+0000","last_active":"2024-09-05T13:12:42.120216+0000","last_peered":"2024-09-05T13:12:42.120216+0000","last_clean":"2024-09-05T13:11:52.103995+0000","last_became_active":"2024-09-05T13:12:25.164969+0000","last_became_peered":"2024-09-05T13:12:25.164969+0000","last_unstale":"2024-09-05T13:12:42.120216+0000","last_undegraded":"2024-09-05T13:12:25.169872+0000","last_fullsized":"2024-09-05T13:12:42.120216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1565,"log_dups_size":0,"ondisk_log_size":1565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6394822,"num_objects":1563,"num_object_clones":0,"num_object_copies":3126,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1565,"num_write_kb":6247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'1465","reported_seq":1521,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:41.062360+0000","last_change":"2024-09-05T13:12:24.168530+0000","last_active":"2024-09-05T13:12:41.062360+0000","last_peered":"2024-09-05T13:12:41.062360+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:24.168300+0000","last_became_peered":"2024-09-05T13:12:24.168300+0000","last_unstale":"2024-09-05T13:12:41.062360+0000","last_undegraded":"2024-09-05T13:12:24.158160+0000","last_fullsized":"2024-09-05T13:12:41.062360+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1465,"log_dups_size":0,"ondisk_log_size":1465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5997027,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":5857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162822+0000","last_change":"2024-09-05T13:12:24.163476+0000","last_active":"2024-09-05T13:12:25.162822+0000","last_peered":"2024-09-05T13:12:25.162822+0000","last_clean":"2024-09-05T13:12:25.162822+0000","last_became_active":"2024-09-05T13:12:24.163169+0000","last_became_peered":"2024-09-05T13:12:24.163169+0000","last_unstale":"2024-09-05T13:12:25.162822+0000","last_undegraded":"2024-09-05T13:12:25.162822+0000","last_fullsized":"2024-09-05T13:12:25.162822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:05:50.037488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'1465","reported_seq":1522,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:41.076678+0000","last_change":"2024-09-05T13:12:24.168576+0000","last_active":"2024-09-05T13:12:41.076678+0000","last_peered":"2024-09-05T13:12:41.076678+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:24.168347+0000","last_became_peered":"2024-09-05T13:12:24.168347+0000","last_unstale":"2024-09-05T13:12:41.076678+0000","last_undegraded":"2024-09-05T13:12:24.157388+0000","last_fullsized":"2024-09-05T13:12:41.076678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1465,"log_dups_size":0,"ondisk_log_size":1465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5981609,"num_objects":1463,"num_object_clones":0,"num_object_copies":2926,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":5844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.166730+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:12:25.166730+0000","last_peered":"2024-09-05T13:12:25.166730+0000","last_clean":"2024-09-05T13:12:25.166730+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:12:25.166730+0000","last_undegraded":"2024-09-05T13:12:25.166730+0000","last_fullsized":"2024-09-05T13:12:25.166730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'1467","reported_seq":1523,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:41.069180+0000","last_change":"2024-09-05T13:12:24.165983+0000","last_active":"2024-09-05T13:12:41.069180+0000","last_peered":"2024-09-05T13:12:41.069180+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:24.165544+0000","last_became_peered":"2024-09-05T13:12:24.165544+0000","last_unstale":"2024-09-05T13:12:41.069180+0000","last_undegraded":"2024-09-05T13:12:24.157734+0000","last_fullsized":"2024-09-05T13:12:41.069180+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1467,"log_dups_size":0,"ondisk_log_size":1467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6001606,"num_objects":1467,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1467,"num_write_kb":5862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1467}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162476+0000","last_change":"2024-09-05T13:12:24.165771+0000","last_active":"2024-09-05T13:12:25.162476+0000","last_peered":"2024-09-05T13:12:25.162476+0000","last_clean":"2024-09-05T13:12:25.162476+0000","last_became_active":"2024-09-05T13:12:24.165476+0000","last_became_peered":"2024-09-05T13:12:24.165476+0000","last_unstale":"2024-09-05T13:12:25.162476+0000","last_undegraded":"2024-09-05T13:12:25.162476+0000","last_fullsized":"2024-09-05T13:12:25.162476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:24.468878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.168816+0000","last_change":"2024-09-05T13:12:25.166728+0000","last_active":"2024-09-05T13:12:25.168816+0000","last_peered":"2024-09-05T13:12:25.168816+0000","last_clean":"2024-09-05T13:12:25.168816+0000","last_became_active":"2024-09-05T13:12:25.166434+0000","last_became_peered":"2024-09-05T13:12:25.166434+0000","last_unstale":"2024-09-05T13:12:25.168816+0000","last_undegraded":"2024-09-05T13:12:25.168816+0000","last_fullsized":"2024-09-05T13:12:25.168816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:26:25.045325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'1431","reported_seq":1489,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:41.075542+0000","last_change":"2024-09-05T13:12:24.164240+0000","last_active":"2024-09-05T13:12:41.075542+0000","last_peered":"2024-09-05T13:12:41.075542+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:24.163127+0000","last_became_peered":"2024-09-05T13:12:24.163127+0000","last_unstale":"2024-09-05T13:12:41.075542+0000","last_undegraded":"2024-09-05T13:12:24.155686+0000","last_fullsized":"2024-09-05T13:12:41.075542+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":1431,"log_dups_size":0,"ondisk_log_size":1431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5826444,"num_objects":1426,"num_object_clones":0,"num_object_copies":2852,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1431,"num_write_kb":5695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1425}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":97957076,"num_objects":23940,"num_object_clones":0,"num_object_copies":47880,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":23940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23967,"num_write_kb":95694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198479872,"data_stored":198342578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23967,"ondisk_log_size":23967,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986184,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":169652,"kb_used_data":93540,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94202188,"statfs":{"total":96636764160,"available":96463040512,"internally_reserved":0,"allocated":95784960,"data_stored":95576887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":77917869},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182412,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":156348,"kb_used_data":84456,"kb_used_omap":4,"kb_used_meta":71867,"kb_avail":94215492,"statfs":{"total":96636764160,"available":96476663808,"internally_reserved":0,"allocated":86482944,"data_stored":86234664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":73592490},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,5,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94138368,"data_stored":94102238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104226816,"data_stored":104226816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:12:43.697 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.f 2024-09-05T13:12:43.697 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-05T13:12:43.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-05T13:12:43.922 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:43.923 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_4", 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 1224592654, 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'3", 2024-09-05T13:12:43.924 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'1", 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_6", 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 3101443614, 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:43.925 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'4", 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'2", 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "NotRecovering", 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:43.926 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:43.927 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:43.927 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:43.946 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.e on osd.2 2024-09-05T13:12:43.946 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost delete 2024-09-05T13:12:44.177 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:44.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 11 20 29759 29739 10.5596 7.41406 0.00956378 0.00739111 2024-09-05T13:12:44.197 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-05T13:12:44.197 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-05T13:12:44.421 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:44.421 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:44.421 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 1, 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 1, 2024-09-05T13:12:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_7", 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 2358579165, 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:44.423 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'3", 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "NotRecovering", 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:44.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:44.425 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:44.425 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:44.425 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:44.425 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:44.425 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:44.425 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:44.445 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.d on osd.2 2024-09-05T13:12:44.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost delete 2024-09-05T13:12:44.702 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-05T13:12:44.720 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.c 2024-09-05T13:12:44.720 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-05T13:12:44.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-05T13:12:44.935 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:44.935 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:44.935 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_3", 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 943960171, 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'5", 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:44.937 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_1", 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 2885884443, 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'4", 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'1", 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:44.939 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:44.956 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.b on osd.0 2024-09-05T13:12:44.957 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost delete 2024-09-05T13:12:45.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 12 20 31526 31506 10.2548 6.90234 0.0120098 0.00760983 2024-09-05T13:12:45.215 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:45.236 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-05T13:12:45.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-05T13:12:45.465 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:45.466 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_9", 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 1217469130, 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'3", 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'1", 2024-09-05T13:12:45.467 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_9", 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 170713514, 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:45.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'2", 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:45.469 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:45.470 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:45.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:45.470 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:45.470 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:45.470 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:45.493 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.a on osd.0 2024-09-05T13:12:45.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost delete 2024-09-05T13:12:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:45.763 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-05T13:12:45.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-05T13:12:45.983 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:45.983 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:45.984 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_5", 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 2998549577, 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'4", 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'2", 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:45.985 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_2", 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 4032093161, 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'3", 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'1", 2024-09-05T13:12:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:45.987 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:45.988 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:46.011 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.9 on osd.0 2024-09-05T13:12:46.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost delete 2024-09-05T13:12:46.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 13 20 33199 33179 9.96864 6.53516 0.0175673 0.00783113 2024-09-05T13:12:46.250 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:46.274 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.8 state active+recovery_wait+degraded 2024-09-05T13:12:46.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-05T13:12:46.493 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:46.493 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:46.493 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:46.493 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:46.493 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:46.493 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:46.493 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 1, 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 1, 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_8", 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 2730327960, 2024-09-05T13:12:46.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'1", 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:46.495 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:46.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:46.496 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:46.496 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:46.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:46.496 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:46.496 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:46.496 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:46.523 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.8 on osd.0 2024-09-05T13:12:46.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost delete 2024-09-05T13:12:46.769 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-05T13:12:46.792 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.0 2024-09-05T13:12:46.792 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-05T13:12:46.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-05T13:12:47.013 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:47.013 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_8", 2024-09-05T13:12:47.014 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 1901787523, 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'5", 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'3", 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_1", 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 3122379363, 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'4", 2024-09-05T13:12:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "NotRecovering", 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:47.018 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:47.037 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.3 on osd.2 2024-09-05T13:12:47.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost delete 2024-09-05T13:12:47.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 14 20 34832 34812 9.7122 6.37891 0.0105391 0.00803747 2024-09-05T13:12:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:47.295 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.1 2024-09-05T13:12:47.296 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-05T13:12:47.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_6", 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 1708673474, 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'5", 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_9", 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 831402034, 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'6", 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:47.541 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.2 on osd.0 2024-09-05T13:12:47.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost delete 2024-09-05T13:12:47.793 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:47.812 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.2 2024-09-05T13:12:47.813 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-05T13:12:47.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-05T13:12:48.042 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:48.042 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_3", 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 459768081, 2024-09-05T13:12:48.043 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'3", 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'1", 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_3", 2024-09-05T13:12:48.044 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 480221105, 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'2", 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:48.045 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "NotRecovering", 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:48.046 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:48.067 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.1 on osd.2 2024-09-05T13:12:48.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost delete 2024-09-05T13:12:48.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 15 20 36521 36501 9.50453 6.59766 0.0148497 0.00821299 2024-09-05T13:12:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:48.324 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:12:48.323+0000 7f9a8a3cf640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 3 pgs recovery_unfound (PG_DAMAGED) 2024-09-05T13:12:48.338 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.3 2024-09-05T13:12:48.338 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-05T13:12:48.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-05T13:12:48.557 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:48.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_5", 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 4248945696, 2024-09-05T13:12:48.559 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'3", 2024-09-05T13:12:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:48.560 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_5", 2024-09-05T13:12:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:48.561 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 4212789968, 2024-09-05T13:12:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'4", 2024-09-05T13:12:48.562 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "NotRecovering", 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:48.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:48.564 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:48.564 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:48.582 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.0 on osd.2 2024-09-05T13:12:48.583 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost delete 2024-09-05T13:12:48.823 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:48.844 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-05T13:12:48.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-05T13:12:49.071 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 1, 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 1, 2024-09-05T13:12:49.072 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_7", 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 3761017527, 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'1", 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:49.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:49.074 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:49.075 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:49.075 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:49.075 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:49.097 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.7 on osd.0 2024-09-05T13:12:49.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost delete 2024-09-05T13:12:49.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 16 20 38070 38050 9.28867 6.05078 0.00968621 0.00840445 2024-09-05T13:12:49.347 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-05T13:12:49.379 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.4 2024-09-05T13:12:49.379 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-05T13:12:49.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-05T13:12:49.611 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:49.611 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:49.611 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:49.611 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:49.611 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:49.611 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 3, 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 3, 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_8", 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:49.612 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 1861424966, 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'5", 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:49.613 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_6", 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 3910131350, 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'4", 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:49.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_4", 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 2787839990, 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:49.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'3", 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:49.616 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:49.617 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:49.617 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:49.617 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:49.629 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.6 on osd.0 2024-09-05T13:12:49.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost delete 2024-09-05T13:12:49.861 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-05T13:12:49.886 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.5 2024-09-05T13:12:49.886 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-05T13:12:49.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-05T13:12:50.100 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:50.100 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:50.100 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:50.100 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:50.100 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:50.100 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:50.100 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:50.100 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 2, 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 2, 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_1", 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 26281365, 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:50.101 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'1", 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "new_2", 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:50.102 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 3196340981, 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'2", 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:50.103 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:50.104 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:50.104 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:50.104 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:50.104 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:50.104 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:50.104 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:50.121 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.5 on osd.0 2024-09-05T13:12:50.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost delete 2024-09-05T13:12:50.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 17 20 39537 39517 9.07934 5.73047 0.00948626 0.00859738 2024-09-05T13:12:50.383 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-05T13:12:50.406 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.6 2024-09-05T13:12:50.406 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.7 2024-09-05T13:12:50.406 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.4 state active+recovering+degraded 2024-09-05T13:12:50.406 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-05T13:12:50.644 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:12:50.644 INFO:teuthology.orchestra.run.smithi086.stdout: "offset": { 2024-09-05T13:12:50.644 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "", 2024-09-05T13:12:50.644 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": 0, 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 0, 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": -9223372036854775808, 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "num_missing": 4, 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "num_unfound": 3, 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "objects": [ 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:50.645 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_4", 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 873891716, 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'8", 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:50.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existing_7", 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 3996155460, 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'9", 2024-09-05T13:12:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "21'6", 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": { 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: "oid": "existed_2", 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "", 2024-09-05T13:12:50.648 INFO:teuthology.orchestra.run.smithi086.stdout: "snapid": -2, 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "hash": 1767667508, 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 0, 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 2, 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "namespace": "" 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "need": "23'7", 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "have": "0'0", 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": "none", 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "locations": [] 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:50.649 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "Active", 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout: "available_might_have_unfound": true, 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout: "might_have_unfound": [ 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "1", 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "osd is down" 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout: "more": false 2024-09-05T13:12:50.650 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:12:50.667 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.4 on osd.0 2024-09-05T13:12:50.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost delete 2024-09-05T13:12:50.904 INFO:teuthology.orchestra.run.smithi086.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-05T13:12:50.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-05T13:12:51.155 INFO:teuthology.orchestra.run.smithi086.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-05T13:12:51.183 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-05T13:12:51.186 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 18 20 41044 41024 8.90195 5.88672 0.0136099 0.00877096 2024-09-05T13:12:51.420 INFO:teuthology.orchestra.run.smithi086.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-05T13:12:51.449 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-05T13:12:51.449 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-05T13:12:51.700 INFO:teuthology.orchestra.run.smithi086.stdout:133143986186 2024-09-05T13:12:51.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:12:51.714 INFO:teuthology.orchestra.run.smithi086.stdout:107374182415 2024-09-05T13:12:51.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:52.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 19 20 42465 42445 8.72555 5.55078 0.0203731 0.00894654 2024-09-05T13:12:52.254 INFO:teuthology.orchestra.run.smithi086.stdout:133143986185 2024-09-05T13:12:52.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986186 got 133143986185 for osd.2 2024-09-05T13:12:52.317 INFO:teuthology.orchestra.run.smithi086.stdout:107374182413 2024-09-05T13:12:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182415 got 107374182413 for osd.0 2024-09-05T13:12:53.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:12:53.186835+0000 min lat: 0.00201413 max lat: 0.0529877 avg lat: 0.00905142 2024-09-05T13:12:53.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:12:53.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 20 20 44191 44171 8.62637 6.74219 0.0109293 0.00905142 2024-09-05T13:12:53.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-05T13:12:53.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-05T13:12:53.845 INFO:teuthology.orchestra.run.smithi086.stdout:133143986187 2024-09-05T13:12:53.867 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986186 got 133143986187 for osd.2 2024-09-05T13:12:53.867 DEBUG:teuthology.parallel:result is None 2024-09-05T13:12:53.911 INFO:teuthology.orchestra.run.smithi086.stdout:107374182415 2024-09-05T13:12:53.938 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182415 got 107374182415 for osd.0 2024-09-05T13:12:53.938 DEBUG:teuthology.parallel:result is None 2024-09-05T13:12:53.938 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-05T13:12:53.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:54.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 21 19 45605 45586 8.47877 5.52734 0.00888989 0.00920723 2024-09-05T13:12:54.473 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:54.494 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-05T13:12:53.514886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":171996140,"num_objects":42010,"num_object_clones":0,"num_object_copies":84020,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30634,"num_write_kb":122416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42074,"ondisk_log_size":42074,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":548856,"kb_used_data":328664,"kb_used_omap":8,"kb_used_meta":220151,"kb_avail":188194824,"statfs":{"total":193273528320,"available":192711499776,"internally_reserved":0,"allocated":336551936,"data_stored":336110035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":225434967},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,8,5,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":94301957,"num_objects":23015,"num_object_clones":0,"num_object_copies":46030,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":23015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11612,"num_write_kb":46508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002880"},"pg_stats":[{"pgid":"2.f","version":"36'2541","reported_seq":2592,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:51.647315+0000","last_change":"2024-09-05T13:12:24.168600+0000","last_active":"2024-09-05T13:12:51.647315+0000","last_peered":"2024-09-05T13:12:51.647315+0000","last_clean":"2024-09-05T13:12:51.647315+0000","last_became_active":"2024-09-05T13:12:24.168356+0000","last_became_peered":"2024-09-05T13:12:24.168356+0000","last_unstale":"2024-09-05T13:12:51.647315+0000","last_undegraded":"2024-09-05T13:12:51.647315+0000","last_fullsized":"2024-09-05T13:12:51.647315+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:44.376826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10407936,"num_objects":2541,"num_object_clones":0,"num_object_copies":5082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2541,"num_write_kb":10164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2636","reported_seq":2721,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.120018+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:12:52.120018+0000","last_peered":"2024-09-05T13:12:52.120018+0000","last_clean":"2024-09-05T13:12:52.120018+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:12:52.120018+0000","last_undegraded":"2024-09-05T13:12:52.120018+0000","last_fullsized":"2024-09-05T13:12:52.120018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":2636,"log_dups_size":0,"ondisk_log_size":2636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10772480,"num_objects":2630,"num_object_clones":0,"num_object_copies":5260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":2584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2738","reported_seq":2840,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.109104+0000","last_change":"2024-09-05T13:12:47.044143+0000","last_active":"2024-09-05T13:12:52.109104+0000","last_peered":"2024-09-05T13:12:52.109104+0000","last_clean":"2024-09-05T13:12:52.109104+0000","last_became_active":"2024-09-05T13:12:25.166237+0000","last_became_peered":"2024-09-05T13:12:25.166237+0000","last_unstale":"2024-09-05T13:12:52.109104+0000","last_undegraded":"2024-09-05T13:12:52.109104+0000","last_fullsized":"2024-09-05T13:12:52.109104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":2738,"log_dups_size":0,"ondisk_log_size":2738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:32:20.708885+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11198464,"num_objects":2734,"num_object_clones":0,"num_object_copies":5468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":549,"num_write_kb":2196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2606","reported_seq":2656,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.106860+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:12:52.106860+0000","last_peered":"2024-09-05T13:12:52.106860+0000","last_clean":"2024-09-05T13:12:52.106860+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:12:52.106860+0000","last_undegraded":"2024-09-05T13:12:52.106860+0000","last_fullsized":"2024-09-05T13:12:52.106860+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2606,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2624","reported_seq":2683,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.656014+0000","last_change":"2024-09-05T13:12:24.165681+0000","last_active":"2024-09-05T13:12:51.656014+0000","last_peered":"2024-09-05T13:12:51.656014+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:12:51.656014+0000","last_undegraded":"2024-09-05T13:12:24.156268+0000","last_fullsized":"2024-09-05T13:12:51.656014+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2624,"log_dups_size":0,"ondisk_log_size":2624,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10720198,"num_objects":2619,"num_object_clones":0,"num_object_copies":5238,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2619}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2614","reported_seq":2670,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.651582+0000","last_change":"2024-09-05T13:12:24.165709+0000","last_active":"2024-09-05T13:12:51.651582+0000","last_peered":"2024-09-05T13:12:51.651582+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:24.165469+0000","last_became_peered":"2024-09-05T13:12:24.165469+0000","last_unstale":"2024-09-05T13:12:51.651582+0000","last_undegraded":"2024-09-05T13:12:24.157014+0000","last_fullsized":"2024-09-05T13:12:51.651582+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2614,"log_dups_size":0,"ondisk_log_size":2614,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10687430,"num_objects":2611,"num_object_clones":0,"num_object_copies":5222,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":10439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2601","reported_seq":2658,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.600525+0000","last_change":"2024-09-05T13:12:24.165854+0000","last_active":"2024-09-05T13:12:51.600525+0000","last_peered":"2024-09-05T13:12:51.600525+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:24.165604+0000","last_became_peered":"2024-09-05T13:12:24.165604+0000","last_unstale":"2024-09-05T13:12:51.600525+0000","last_undegraded":"2024-09-05T13:12:24.157554+0000","last_fullsized":"2024-09-05T13:12:51.600525+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10630086,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2599,"num_write_kb":10384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2597}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2647","reported_seq":2708,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.642482+0000","last_change":"2024-09-05T13:12:24.163654+0000","last_active":"2024-09-05T13:12:51.642482+0000","last_peered":"2024-09-05T13:12:51.642482+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:12:51.642482+0000","last_undegraded":"2024-09-05T13:12:24.156202+0000","last_fullsized":"2024-09-05T13:12:51.642482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2647,"log_dups_size":0,"ondisk_log_size":2647,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10834403,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2646,"num_write_kb":10581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2646}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.160861+0000","last_change":"2024-09-05T13:12:24.163255+0000","last_active":"2024-09-05T13:12:25.160861+0000","last_peered":"2024-09-05T13:12:25.160861+0000","last_clean":"2024-09-05T13:12:25.160861+0000","last_became_active":"2024-09-05T13:12:24.162958+0000","last_became_peered":"2024-09-05T13:12:24.162958+0000","last_unstale":"2024-09-05T13:12:25.160861+0000","last_undegraded":"2024-09-05T13:12:25.160861+0000","last_fullsized":"2024-09-05T13:12:25.160861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:58.405279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2641","reported_seq":2745,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.106252+0000","last_change":"2024-09-05T13:12:49.050077+0000","last_active":"2024-09-05T13:12:52.106252+0000","last_peered":"2024-09-05T13:12:52.106252+0000","last_clean":"2024-09-05T13:12:52.106252+0000","last_became_active":"2024-09-05T13:12:25.167319+0000","last_became_peered":"2024-09-05T13:12:25.167319+0000","last_unstale":"2024-09-05T13:12:52.106252+0000","last_undegraded":"2024-09-05T13:12:52.106252+0000","last_fullsized":"2024-09-05T13:12:52.106252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":2641,"log_dups_size":0,"ondisk_log_size":2641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:33.891462+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10788864,"num_objects":2634,"num_object_clones":0,"num_object_copies":5268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":295,"num_write_kb":1180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.165344+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:12:25.165344+0000","last_peered":"2024-09-05T13:12:25.165344+0000","last_clean":"2024-09-05T13:12:25.165344+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:12:25.165344+0000","last_undegraded":"2024-09-05T13:12:25.165344+0000","last_fullsized":"2024-09-05T13:12:25.165344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2601","reported_seq":2651,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.665327+0000","last_change":"2024-09-05T13:12:24.165760+0000","last_active":"2024-09-05T13:12:51.665327+0000","last_peered":"2024-09-05T13:12:51.665327+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:24.165511+0000","last_became_peered":"2024-09-05T13:12:24.165511+0000","last_unstale":"2024-09-05T13:12:51.665327+0000","last_undegraded":"2024-09-05T13:12:24.157434+0000","last_fullsized":"2024-09-05T13:12:51.665327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10621894,"num_objects":2595,"num_object_clones":0,"num_object_copies":5190,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2599,"num_write_kb":10376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162067+0000","last_change":"2024-09-05T13:12:24.164231+0000","last_active":"2024-09-05T13:12:25.162067+0000","last_peered":"2024-09-05T13:12:25.162067+0000","last_clean":"2024-09-05T13:12:03.089565+0000","last_became_active":"2024-09-05T13:12:24.163048+0000","last_became_peered":"2024-09-05T13:12:24.163048+0000","last_unstale":"2024-09-05T13:12:25.162067+0000","last_undegraded":"2024-09-05T13:12:24.156585+0000","last_fullsized":"2024-09-05T13:12:25.162067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2627","reported_seq":2720,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.134273+0000","last_change":"2024-09-05T13:12:50.064282+0000","last_active":"2024-09-05T13:12:52.134273+0000","last_peered":"2024-09-05T13:12:52.134273+0000","last_clean":"2024-09-05T13:12:52.134273+0000","last_became_active":"2024-09-05T13:12:25.165724+0000","last_became_peered":"2024-09-05T13:12:25.165724+0000","last_unstale":"2024-09-05T13:12:52.134273+0000","last_undegraded":"2024-09-05T13:12:52.134273+0000","last_fullsized":"2024-09-05T13:12:52.134273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":2627,"log_dups_size":0,"ondisk_log_size":2627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:47:05.084065+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10739712,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.169048+0000","last_change":"2024-09-05T13:12:25.167242+0000","last_active":"2024-09-05T13:12:25.169048+0000","last_peered":"2024-09-05T13:12:25.169048+0000","last_clean":"2024-09-05T13:12:25.169048+0000","last_became_active":"2024-09-05T13:12:25.165964+0000","last_became_peered":"2024-09-05T13:12:25.165964+0000","last_unstale":"2024-09-05T13:12:25.169048+0000","last_undegraded":"2024-09-05T13:12:25.169048+0000","last_fullsized":"2024-09-05T13:12:25.169048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:03.155260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2623","reported_seq":2719,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.112926+0000","last_change":"2024-09-05T13:12:51.179418+0000","last_active":"2024-09-05T13:12:52.112926+0000","last_peered":"2024-09-05T13:12:52.112926+0000","last_clean":"2024-09-05T13:12:52.112926+0000","last_became_active":"2024-09-05T13:12:25.164969+0000","last_became_peered":"2024-09-05T13:12:25.164969+0000","last_unstale":"2024-09-05T13:12:52.112926+0000","last_undegraded":"2024-09-05T13:12:52.112926+0000","last_fullsized":"2024-09-05T13:12:52.112926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":2623,"log_dups_size":0,"ondisk_log_size":2623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:28:28.361737+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10719232,"num_objects":2617,"num_object_clones":0,"num_object_copies":5234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2613","reported_seq":2666,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.673721+0000","last_change":"2024-09-05T13:12:24.168530+0000","last_active":"2024-09-05T13:12:51.673721+0000","last_peered":"2024-09-05T13:12:51.673721+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:24.168300+0000","last_became_peered":"2024-09-05T13:12:24.168300+0000","last_unstale":"2024-09-05T13:12:51.673721+0000","last_undegraded":"2024-09-05T13:12:24.158160+0000","last_fullsized":"2024-09-05T13:12:51.673721+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2613,"log_dups_size":0,"ondisk_log_size":2613,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10695139,"num_objects":2612,"num_object_clones":0,"num_object_copies":5224,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2612}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162822+0000","last_change":"2024-09-05T13:12:24.163476+0000","last_active":"2024-09-05T13:12:25.162822+0000","last_peered":"2024-09-05T13:12:25.162822+0000","last_clean":"2024-09-05T13:12:25.162822+0000","last_became_active":"2024-09-05T13:12:24.163169+0000","last_became_peered":"2024-09-05T13:12:24.163169+0000","last_unstale":"2024-09-05T13:12:25.162822+0000","last_undegraded":"2024-09-05T13:12:25.162822+0000","last_fullsized":"2024-09-05T13:12:25.162822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:05:50.037488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2619","reported_seq":2674,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.682073+0000","last_change":"2024-09-05T13:12:24.168576+0000","last_active":"2024-09-05T13:12:51.682073+0000","last_peered":"2024-09-05T13:12:51.682073+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:24.168347+0000","last_became_peered":"2024-09-05T13:12:24.168347+0000","last_unstale":"2024-09-05T13:12:51.682073+0000","last_undegraded":"2024-09-05T13:12:24.157388+0000","last_fullsized":"2024-09-05T13:12:51.682073+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2619,"log_dups_size":0,"ondisk_log_size":2619,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10696105,"num_objects":2614,"num_object_clones":0,"num_object_copies":5228,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2616,"num_write_kb":10448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.166730+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:12:25.166730+0000","last_peered":"2024-09-05T13:12:25.166730+0000","last_clean":"2024-09-05T13:12:25.166730+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:12:25.166730+0000","last_undegraded":"2024-09-05T13:12:25.166730+0000","last_fullsized":"2024-09-05T13:12:25.166730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2680","reported_seq":2734,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.669971+0000","last_change":"2024-09-05T13:12:24.165983+0000","last_active":"2024-09-05T13:12:51.669971+0000","last_peered":"2024-09-05T13:12:51.669971+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:24.165544+0000","last_became_peered":"2024-09-05T13:12:24.165544+0000","last_unstale":"2024-09-05T13:12:51.669971+0000","last_undegraded":"2024-09-05T13:12:24.157734+0000","last_fullsized":"2024-09-05T13:12:51.669971+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2680,"log_dups_size":0,"ondisk_log_size":2680,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10961862,"num_objects":2678,"num_object_clones":0,"num_object_copies":5356,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2678,"num_write_kb":10706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2678}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162476+0000","last_change":"2024-09-05T13:12:24.165771+0000","last_active":"2024-09-05T13:12:25.162476+0000","last_peered":"2024-09-05T13:12:25.162476+0000","last_clean":"2024-09-05T13:12:25.162476+0000","last_became_active":"2024-09-05T13:12:24.165476+0000","last_became_peered":"2024-09-05T13:12:24.165476+0000","last_unstale":"2024-09-05T13:12:25.162476+0000","last_undegraded":"2024-09-05T13:12:25.162476+0000","last_fullsized":"2024-09-05T13:12:25.162476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:24.468878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.168816+0000","last_change":"2024-09-05T13:12:25.166728+0000","last_active":"2024-09-05T13:12:25.168816+0000","last_peered":"2024-09-05T13:12:25.168816+0000","last_clean":"2024-09-05T13:12:25.168816+0000","last_became_active":"2024-09-05T13:12:25.166434+0000","last_became_peered":"2024-09-05T13:12:25.166434+0000","last_unstale":"2024-09-05T13:12:25.168816+0000","last_undegraded":"2024-09-05T13:12:25.168816+0000","last_fullsized":"2024-09-05T13:12:25.168816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:26:25.045325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2660","reported_seq":2716,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:51.674293+0000","last_change":"2024-09-05T13:12:24.164240+0000","last_active":"2024-09-05T13:12:51.674293+0000","last_peered":"2024-09-05T13:12:51.674293+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:24.163127+0000","last_became_peered":"2024-09-05T13:12:24.163127+0000","last_unstale":"2024-09-05T13:12:51.674293+0000","last_undegraded":"2024-09-05T13:12:24.155686+0000","last_fullsized":"2024-09-05T13:12:51.674293+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2660,"log_dups_size":0,"ondisk_log_size":2660,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10848140,"num_objects":2652,"num_object_clones":0,"num_object_copies":5304,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2657,"num_write_kb":10599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":171996121,"num_objects":42008,"num_object_clones":0,"num_object_copies":84016,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30632,"num_write_kb":122414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344932352,"data_stored":344809510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42071,"ondisk_log_size":42071,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276624,"kb_used_data":164416,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94095216,"statfs":{"total":96636764160,"available":96353501184,"internally_reserved":0,"allocated":168361984,"data_stored":168153911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":114880173},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272232,"kb_used_data":164248,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94099608,"statfs":{"total":96636764160,"available":96357998592,"internally_reserved":0,"allocated":168189952,"data_stored":167956124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110554794},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171151360,"data_stored":171129682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":173666304,"data_stored":173666304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:12:54.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:55.034 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:55.034 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:55.057 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-05T13:12:53.514886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":171996140,"num_objects":42010,"num_object_clones":0,"num_object_copies":84020,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30634,"num_write_kb":122416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42074,"ondisk_log_size":42074,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":548856,"kb_used_data":328664,"kb_used_omap":8,"kb_used_meta":220151,"kb_avail":188194824,"statfs":{"total":193273528320,"available":192711499776,"internally_reserved":0,"allocated":336551936,"data_stored":336110035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":225434967},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,8,5,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":94301957,"num_objects":23015,"num_object_clones":0,"num_object_copies":46030,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":23015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11612,"num_write_kb":46508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002880"},"pg_stats":[{"pgid":"2.f","version":"36'2541","reported_seq":2592,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:51.647315+0000","last_change":"2024-09-05T13:12:24.168600+0000","last_active":"2024-09-05T13:12:51.647315+0000","last_peered":"2024-09-05T13:12:51.647315+0000","last_clean":"2024-09-05T13:12:51.647315+0000","last_became_active":"2024-09-05T13:12:24.168356+0000","last_became_peered":"2024-09-05T13:12:24.168356+0000","last_unstale":"2024-09-05T13:12:51.647315+0000","last_undegraded":"2024-09-05T13:12:51.647315+0000","last_fullsized":"2024-09-05T13:12:51.647315+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:44.376826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10407936,"num_objects":2541,"num_object_clones":0,"num_object_copies":5082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2541,"num_write_kb":10164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2636","reported_seq":2721,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.120018+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:12:52.120018+0000","last_peered":"2024-09-05T13:12:52.120018+0000","last_clean":"2024-09-05T13:12:52.120018+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:12:52.120018+0000","last_undegraded":"2024-09-05T13:12:52.120018+0000","last_fullsized":"2024-09-05T13:12:52.120018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":2636,"log_dups_size":0,"ondisk_log_size":2636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10772480,"num_objects":2630,"num_object_clones":0,"num_object_copies":5260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":2584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2738","reported_seq":2840,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.109104+0000","last_change":"2024-09-05T13:12:47.044143+0000","last_active":"2024-09-05T13:12:52.109104+0000","last_peered":"2024-09-05T13:12:52.109104+0000","last_clean":"2024-09-05T13:12:52.109104+0000","last_became_active":"2024-09-05T13:12:25.166237+0000","last_became_peered":"2024-09-05T13:12:25.166237+0000","last_unstale":"2024-09-05T13:12:52.109104+0000","last_undegraded":"2024-09-05T13:12:52.109104+0000","last_fullsized":"2024-09-05T13:12:52.109104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":2738,"log_dups_size":0,"ondisk_log_size":2738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:32:20.708885+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11198464,"num_objects":2734,"num_object_clones":0,"num_object_copies":5468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":549,"num_write_kb":2196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2606","reported_seq":2656,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.106860+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:12:52.106860+0000","last_peered":"2024-09-05T13:12:52.106860+0000","last_clean":"2024-09-05T13:12:52.106860+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:12:52.106860+0000","last_undegraded":"2024-09-05T13:12:52.106860+0000","last_fullsized":"2024-09-05T13:12:52.106860+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2606,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2624","reported_seq":2683,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.656014+0000","last_change":"2024-09-05T13:12:24.165681+0000","last_active":"2024-09-05T13:12:51.656014+0000","last_peered":"2024-09-05T13:12:51.656014+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:12:51.656014+0000","last_undegraded":"2024-09-05T13:12:24.156268+0000","last_fullsized":"2024-09-05T13:12:51.656014+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2624,"log_dups_size":0,"ondisk_log_size":2624,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10720198,"num_objects":2619,"num_object_clones":0,"num_object_copies":5238,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2619}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2614","reported_seq":2670,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.651582+0000","last_change":"2024-09-05T13:12:24.165709+0000","last_active":"2024-09-05T13:12:51.651582+0000","last_peered":"2024-09-05T13:12:51.651582+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:24.165469+0000","last_became_peered":"2024-09-05T13:12:24.165469+0000","last_unstale":"2024-09-05T13:12:51.651582+0000","last_undegraded":"2024-09-05T13:12:24.157014+0000","last_fullsized":"2024-09-05T13:12:51.651582+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2614,"log_dups_size":0,"ondisk_log_size":2614,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10687430,"num_objects":2611,"num_object_clones":0,"num_object_copies":5222,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":10439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2601","reported_seq":2658,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.600525+0000","last_change":"2024-09-05T13:12:24.165854+0000","last_active":"2024-09-05T13:12:51.600525+0000","last_peered":"2024-09-05T13:12:51.600525+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:24.165604+0000","last_became_peered":"2024-09-05T13:12:24.165604+0000","last_unstale":"2024-09-05T13:12:51.600525+0000","last_undegraded":"2024-09-05T13:12:24.157554+0000","last_fullsized":"2024-09-05T13:12:51.600525+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10630086,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2599,"num_write_kb":10384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2597}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2647","reported_seq":2708,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.642482+0000","last_change":"2024-09-05T13:12:24.163654+0000","last_active":"2024-09-05T13:12:51.642482+0000","last_peered":"2024-09-05T13:12:51.642482+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:12:51.642482+0000","last_undegraded":"2024-09-05T13:12:24.156202+0000","last_fullsized":"2024-09-05T13:12:51.642482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2647,"log_dups_size":0,"ondisk_log_size":2647,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10834403,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2646,"num_write_kb":10581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2646}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.160861+0000","last_change":"2024-09-05T13:12:24.163255+0000","last_active":"2024-09-05T13:12:25.160861+0000","last_peered":"2024-09-05T13:12:25.160861+0000","last_clean":"2024-09-05T13:12:25.160861+0000","last_became_active":"2024-09-05T13:12:24.162958+0000","last_became_peered":"2024-09-05T13:12:24.162958+0000","last_unstale":"2024-09-05T13:12:25.160861+0000","last_undegraded":"2024-09-05T13:12:25.160861+0000","last_fullsized":"2024-09-05T13:12:25.160861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:58.405279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2641","reported_seq":2745,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.106252+0000","last_change":"2024-09-05T13:12:49.050077+0000","last_active":"2024-09-05T13:12:52.106252+0000","last_peered":"2024-09-05T13:12:52.106252+0000","last_clean":"2024-09-05T13:12:52.106252+0000","last_became_active":"2024-09-05T13:12:25.167319+0000","last_became_peered":"2024-09-05T13:12:25.167319+0000","last_unstale":"2024-09-05T13:12:52.106252+0000","last_undegraded":"2024-09-05T13:12:52.106252+0000","last_fullsized":"2024-09-05T13:12:52.106252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":2641,"log_dups_size":0,"ondisk_log_size":2641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:33.891462+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10788864,"num_objects":2634,"num_object_clones":0,"num_object_copies":5268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":295,"num_write_kb":1180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.165344+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:12:25.165344+0000","last_peered":"2024-09-05T13:12:25.165344+0000","last_clean":"2024-09-05T13:12:25.165344+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:12:25.165344+0000","last_undegraded":"2024-09-05T13:12:25.165344+0000","last_fullsized":"2024-09-05T13:12:25.165344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2601","reported_seq":2651,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.665327+0000","last_change":"2024-09-05T13:12:24.165760+0000","last_active":"2024-09-05T13:12:51.665327+0000","last_peered":"2024-09-05T13:12:51.665327+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:24.165511+0000","last_became_peered":"2024-09-05T13:12:24.165511+0000","last_unstale":"2024-09-05T13:12:51.665327+0000","last_undegraded":"2024-09-05T13:12:24.157434+0000","last_fullsized":"2024-09-05T13:12:51.665327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10621894,"num_objects":2595,"num_object_clones":0,"num_object_copies":5190,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2599,"num_write_kb":10376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162067+0000","last_change":"2024-09-05T13:12:24.164231+0000","last_active":"2024-09-05T13:12:25.162067+0000","last_peered":"2024-09-05T13:12:25.162067+0000","last_clean":"2024-09-05T13:12:03.089565+0000","last_became_active":"2024-09-05T13:12:24.163048+0000","last_became_peered":"2024-09-05T13:12:24.163048+0000","last_unstale":"2024-09-05T13:12:25.162067+0000","last_undegraded":"2024-09-05T13:12:24.156585+0000","last_fullsized":"2024-09-05T13:12:25.162067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2627","reported_seq":2720,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.134273+0000","last_change":"2024-09-05T13:12:50.064282+0000","last_active":"2024-09-05T13:12:52.134273+0000","last_peered":"2024-09-05T13:12:52.134273+0000","last_clean":"2024-09-05T13:12:52.134273+0000","last_became_active":"2024-09-05T13:12:25.165724+0000","last_became_peered":"2024-09-05T13:12:25.165724+0000","last_unstale":"2024-09-05T13:12:52.134273+0000","last_undegraded":"2024-09-05T13:12:52.134273+0000","last_fullsized":"2024-09-05T13:12:52.134273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":2627,"log_dups_size":0,"ondisk_log_size":2627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:47:05.084065+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10739712,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.169048+0000","last_change":"2024-09-05T13:12:25.167242+0000","last_active":"2024-09-05T13:12:25.169048+0000","last_peered":"2024-09-05T13:12:25.169048+0000","last_clean":"2024-09-05T13:12:25.169048+0000","last_became_active":"2024-09-05T13:12:25.165964+0000","last_became_peered":"2024-09-05T13:12:25.165964+0000","last_unstale":"2024-09-05T13:12:25.169048+0000","last_undegraded":"2024-09-05T13:12:25.169048+0000","last_fullsized":"2024-09-05T13:12:25.169048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:03.155260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2623","reported_seq":2719,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.112926+0000","last_change":"2024-09-05T13:12:51.179418+0000","last_active":"2024-09-05T13:12:52.112926+0000","last_peered":"2024-09-05T13:12:52.112926+0000","last_clean":"2024-09-05T13:12:52.112926+0000","last_became_active":"2024-09-05T13:12:25.164969+0000","last_became_peered":"2024-09-05T13:12:25.164969+0000","last_unstale":"2024-09-05T13:12:52.112926+0000","last_undegraded":"2024-09-05T13:12:52.112926+0000","last_fullsized":"2024-09-05T13:12:52.112926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":2623,"log_dups_size":0,"ondisk_log_size":2623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:28:28.361737+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10719232,"num_objects":2617,"num_object_clones":0,"num_object_copies":5234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2613","reported_seq":2666,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.673721+0000","last_change":"2024-09-05T13:12:24.168530+0000","last_active":"2024-09-05T13:12:51.673721+0000","last_peered":"2024-09-05T13:12:51.673721+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:24.168300+0000","last_became_peered":"2024-09-05T13:12:24.168300+0000","last_unstale":"2024-09-05T13:12:51.673721+0000","last_undegraded":"2024-09-05T13:12:24.158160+0000","last_fullsized":"2024-09-05T13:12:51.673721+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2613,"log_dups_size":0,"ondisk_log_size":2613,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10695139,"num_objects":2612,"num_object_clones":0,"num_object_copies":5224,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2612}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162822+0000","last_change":"2024-09-05T13:12:24.163476+0000","last_active":"2024-09-05T13:12:25.162822+0000","last_peered":"2024-09-05T13:12:25.162822+0000","last_clean":"2024-09-05T13:12:25.162822+0000","last_became_active":"2024-09-05T13:12:24.163169+0000","last_became_peered":"2024-09-05T13:12:24.163169+0000","last_unstale":"2024-09-05T13:12:25.162822+0000","last_undegraded":"2024-09-05T13:12:25.162822+0000","last_fullsized":"2024-09-05T13:12:25.162822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:05:50.037488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2619","reported_seq":2674,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.682073+0000","last_change":"2024-09-05T13:12:24.168576+0000","last_active":"2024-09-05T13:12:51.682073+0000","last_peered":"2024-09-05T13:12:51.682073+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:24.168347+0000","last_became_peered":"2024-09-05T13:12:24.168347+0000","last_unstale":"2024-09-05T13:12:51.682073+0000","last_undegraded":"2024-09-05T13:12:24.157388+0000","last_fullsized":"2024-09-05T13:12:51.682073+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2619,"log_dups_size":0,"ondisk_log_size":2619,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10696105,"num_objects":2614,"num_object_clones":0,"num_object_copies":5228,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2616,"num_write_kb":10448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.166730+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:12:25.166730+0000","last_peered":"2024-09-05T13:12:25.166730+0000","last_clean":"2024-09-05T13:12:25.166730+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:12:25.166730+0000","last_undegraded":"2024-09-05T13:12:25.166730+0000","last_fullsized":"2024-09-05T13:12:25.166730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2680","reported_seq":2734,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.669971+0000","last_change":"2024-09-05T13:12:24.165983+0000","last_active":"2024-09-05T13:12:51.669971+0000","last_peered":"2024-09-05T13:12:51.669971+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:24.165544+0000","last_became_peered":"2024-09-05T13:12:24.165544+0000","last_unstale":"2024-09-05T13:12:51.669971+0000","last_undegraded":"2024-09-05T13:12:24.157734+0000","last_fullsized":"2024-09-05T13:12:51.669971+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2680,"log_dups_size":0,"ondisk_log_size":2680,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10961862,"num_objects":2678,"num_object_clones":0,"num_object_copies":5356,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2678,"num_write_kb":10706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2678}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162476+0000","last_change":"2024-09-05T13:12:24.165771+0000","last_active":"2024-09-05T13:12:25.162476+0000","last_peered":"2024-09-05T13:12:25.162476+0000","last_clean":"2024-09-05T13:12:25.162476+0000","last_became_active":"2024-09-05T13:12:24.165476+0000","last_became_peered":"2024-09-05T13:12:24.165476+0000","last_unstale":"2024-09-05T13:12:25.162476+0000","last_undegraded":"2024-09-05T13:12:25.162476+0000","last_fullsized":"2024-09-05T13:12:25.162476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:24.468878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.168816+0000","last_change":"2024-09-05T13:12:25.166728+0000","last_active":"2024-09-05T13:12:25.168816+0000","last_peered":"2024-09-05T13:12:25.168816+0000","last_clean":"2024-09-05T13:12:25.168816+0000","last_became_active":"2024-09-05T13:12:25.166434+0000","last_became_peered":"2024-09-05T13:12:25.166434+0000","last_unstale":"2024-09-05T13:12:25.168816+0000","last_undegraded":"2024-09-05T13:12:25.168816+0000","last_fullsized":"2024-09-05T13:12:25.168816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:26:25.045325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2660","reported_seq":2716,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:51.674293+0000","last_change":"2024-09-05T13:12:24.164240+0000","last_active":"2024-09-05T13:12:51.674293+0000","last_peered":"2024-09-05T13:12:51.674293+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:24.163127+0000","last_became_peered":"2024-09-05T13:12:24.163127+0000","last_unstale":"2024-09-05T13:12:51.674293+0000","last_undegraded":"2024-09-05T13:12:24.155686+0000","last_fullsized":"2024-09-05T13:12:51.674293+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2660,"log_dups_size":0,"ondisk_log_size":2660,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10848140,"num_objects":2652,"num_object_clones":0,"num_object_copies":5304,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2657,"num_write_kb":10599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":171996121,"num_objects":42008,"num_object_clones":0,"num_object_copies":84016,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30632,"num_write_kb":122414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344932352,"data_stored":344809510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42071,"ondisk_log_size":42071,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276624,"kb_used_data":164416,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94095216,"statfs":{"total":96636764160,"available":96353501184,"internally_reserved":0,"allocated":168361984,"data_stored":168153911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":114880173},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272232,"kb_used_data":164248,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94099608,"statfs":{"total":96636764160,"available":96357998592,"internally_reserved":0,"allocated":168189952,"data_stored":167956124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110554794},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171151360,"data_stored":171129682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":173666304,"data_stored":173666304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:12:55.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:55.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 22 20 47093 47073 8.35738 5.80859 0.0102803 0.00934104 2024-09-05T13:12:55.603 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:55.604 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:55.621 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-05T13:12:55.515344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":171996140,"num_objects":42010,"num_object_clones":0,"num_object_copies":84020,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30634,"num_write_kb":122416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42074,"ondisk_log_size":42074,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":548856,"kb_used_data":328664,"kb_used_omap":8,"kb_used_meta":220151,"kb_avail":188194824,"statfs":{"total":193273528320,"available":192711499776,"internally_reserved":0,"allocated":336551936,"data_stored":336110035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":225434967},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,8,5,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":74039045,"num_objects":18068,"num_object_clones":0,"num_object_copies":36136,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":18068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002824"},"pg_stats":[{"pgid":"2.f","version":"36'2541","reported_seq":2592,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:51.647315+0000","last_change":"2024-09-05T13:12:24.168600+0000","last_active":"2024-09-05T13:12:51.647315+0000","last_peered":"2024-09-05T13:12:51.647315+0000","last_clean":"2024-09-05T13:12:51.647315+0000","last_became_active":"2024-09-05T13:12:24.168356+0000","last_became_peered":"2024-09-05T13:12:24.168356+0000","last_unstale":"2024-09-05T13:12:51.647315+0000","last_undegraded":"2024-09-05T13:12:51.647315+0000","last_fullsized":"2024-09-05T13:12:51.647315+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:44.376826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10407936,"num_objects":2541,"num_object_clones":0,"num_object_copies":5082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2541,"num_write_kb":10164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2636","reported_seq":2721,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.120018+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:12:52.120018+0000","last_peered":"2024-09-05T13:12:52.120018+0000","last_clean":"2024-09-05T13:12:52.120018+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:12:52.120018+0000","last_undegraded":"2024-09-05T13:12:52.120018+0000","last_fullsized":"2024-09-05T13:12:52.120018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":2636,"log_dups_size":0,"ondisk_log_size":2636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10772480,"num_objects":2630,"num_object_clones":0,"num_object_copies":5260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":646,"num_write_kb":2584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2738","reported_seq":2840,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.109104+0000","last_change":"2024-09-05T13:12:47.044143+0000","last_active":"2024-09-05T13:12:52.109104+0000","last_peered":"2024-09-05T13:12:52.109104+0000","last_clean":"2024-09-05T13:12:52.109104+0000","last_became_active":"2024-09-05T13:12:25.166237+0000","last_became_peered":"2024-09-05T13:12:25.166237+0000","last_unstale":"2024-09-05T13:12:52.109104+0000","last_undegraded":"2024-09-05T13:12:52.109104+0000","last_fullsized":"2024-09-05T13:12:52.109104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":2738,"log_dups_size":0,"ondisk_log_size":2738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:32:20.708885+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11198464,"num_objects":2734,"num_object_clones":0,"num_object_copies":5468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":549,"num_write_kb":2196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2606","reported_seq":2656,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.106860+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:12:52.106860+0000","last_peered":"2024-09-05T13:12:52.106860+0000","last_clean":"2024-09-05T13:12:52.106860+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:12:52.106860+0000","last_undegraded":"2024-09-05T13:12:52.106860+0000","last_fullsized":"2024-09-05T13:12:52.106860+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10674176,"num_objects":2606,"num_object_clones":0,"num_object_copies":5212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2606,"num_write_kb":10424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2624","reported_seq":2683,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.656014+0000","last_change":"2024-09-05T13:12:24.165681+0000","last_active":"2024-09-05T13:12:51.656014+0000","last_peered":"2024-09-05T13:12:51.656014+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:12:51.656014+0000","last_undegraded":"2024-09-05T13:12:24.156268+0000","last_fullsized":"2024-09-05T13:12:51.656014+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2624,"log_dups_size":0,"ondisk_log_size":2624,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10720198,"num_objects":2619,"num_object_clones":0,"num_object_copies":5238,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2619}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2614","reported_seq":2670,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.651582+0000","last_change":"2024-09-05T13:12:24.165709+0000","last_active":"2024-09-05T13:12:51.651582+0000","last_peered":"2024-09-05T13:12:51.651582+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:24.165469+0000","last_became_peered":"2024-09-05T13:12:24.165469+0000","last_unstale":"2024-09-05T13:12:51.651582+0000","last_undegraded":"2024-09-05T13:12:24.157014+0000","last_fullsized":"2024-09-05T13:12:51.651582+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2614,"log_dups_size":0,"ondisk_log_size":2614,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10687430,"num_objects":2611,"num_object_clones":0,"num_object_copies":5222,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":10439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2601","reported_seq":2658,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.600525+0000","last_change":"2024-09-05T13:12:24.165854+0000","last_active":"2024-09-05T13:12:51.600525+0000","last_peered":"2024-09-05T13:12:51.600525+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:24.165604+0000","last_became_peered":"2024-09-05T13:12:24.165604+0000","last_unstale":"2024-09-05T13:12:51.600525+0000","last_undegraded":"2024-09-05T13:12:24.157554+0000","last_fullsized":"2024-09-05T13:12:51.600525+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10630086,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2599,"num_write_kb":10384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2597}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2647","reported_seq":2708,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.642482+0000","last_change":"2024-09-05T13:12:24.163654+0000","last_active":"2024-09-05T13:12:51.642482+0000","last_peered":"2024-09-05T13:12:51.642482+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:12:51.642482+0000","last_undegraded":"2024-09-05T13:12:24.156202+0000","last_fullsized":"2024-09-05T13:12:51.642482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2647,"log_dups_size":0,"ondisk_log_size":2647,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10834403,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2646,"num_write_kb":10581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2646}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.160861+0000","last_change":"2024-09-05T13:12:24.163255+0000","last_active":"2024-09-05T13:12:25.160861+0000","last_peered":"2024-09-05T13:12:25.160861+0000","last_clean":"2024-09-05T13:12:25.160861+0000","last_became_active":"2024-09-05T13:12:24.162958+0000","last_became_peered":"2024-09-05T13:12:24.162958+0000","last_unstale":"2024-09-05T13:12:25.160861+0000","last_undegraded":"2024-09-05T13:12:25.160861+0000","last_fullsized":"2024-09-05T13:12:25.160861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:58.405279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2641","reported_seq":2745,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.106252+0000","last_change":"2024-09-05T13:12:49.050077+0000","last_active":"2024-09-05T13:12:52.106252+0000","last_peered":"2024-09-05T13:12:52.106252+0000","last_clean":"2024-09-05T13:12:52.106252+0000","last_became_active":"2024-09-05T13:12:25.167319+0000","last_became_peered":"2024-09-05T13:12:25.167319+0000","last_unstale":"2024-09-05T13:12:52.106252+0000","last_undegraded":"2024-09-05T13:12:52.106252+0000","last_fullsized":"2024-09-05T13:12:52.106252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":2641,"log_dups_size":0,"ondisk_log_size":2641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:33.891462+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10788864,"num_objects":2634,"num_object_clones":0,"num_object_copies":5268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":295,"num_write_kb":1180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.165344+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:12:25.165344+0000","last_peered":"2024-09-05T13:12:25.165344+0000","last_clean":"2024-09-05T13:12:25.165344+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:12:25.165344+0000","last_undegraded":"2024-09-05T13:12:25.165344+0000","last_fullsized":"2024-09-05T13:12:25.165344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2601","reported_seq":2651,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.665327+0000","last_change":"2024-09-05T13:12:24.165760+0000","last_active":"2024-09-05T13:12:51.665327+0000","last_peered":"2024-09-05T13:12:51.665327+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:24.165511+0000","last_became_peered":"2024-09-05T13:12:24.165511+0000","last_unstale":"2024-09-05T13:12:51.665327+0000","last_undegraded":"2024-09-05T13:12:24.157434+0000","last_fullsized":"2024-09-05T13:12:51.665327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10621894,"num_objects":2595,"num_object_clones":0,"num_object_copies":5190,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2599,"num_write_kb":10376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162067+0000","last_change":"2024-09-05T13:12:24.164231+0000","last_active":"2024-09-05T13:12:25.162067+0000","last_peered":"2024-09-05T13:12:25.162067+0000","last_clean":"2024-09-05T13:12:03.089565+0000","last_became_active":"2024-09-05T13:12:24.163048+0000","last_became_peered":"2024-09-05T13:12:24.163048+0000","last_unstale":"2024-09-05T13:12:25.162067+0000","last_undegraded":"2024-09-05T13:12:24.156585+0000","last_fullsized":"2024-09-05T13:12:25.162067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2627","reported_seq":2720,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.134273+0000","last_change":"2024-09-05T13:12:50.064282+0000","last_active":"2024-09-05T13:12:52.134273+0000","last_peered":"2024-09-05T13:12:52.134273+0000","last_clean":"2024-09-05T13:12:52.134273+0000","last_became_active":"2024-09-05T13:12:25.165724+0000","last_became_peered":"2024-09-05T13:12:25.165724+0000","last_unstale":"2024-09-05T13:12:52.134273+0000","last_undegraded":"2024-09-05T13:12:52.134273+0000","last_fullsized":"2024-09-05T13:12:52.134273+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":2627,"log_dups_size":0,"ondisk_log_size":2627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:47:05.084065+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10739712,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.169048+0000","last_change":"2024-09-05T13:12:25.167242+0000","last_active":"2024-09-05T13:12:25.169048+0000","last_peered":"2024-09-05T13:12:25.169048+0000","last_clean":"2024-09-05T13:12:25.169048+0000","last_became_active":"2024-09-05T13:12:25.165964+0000","last_became_peered":"2024-09-05T13:12:25.165964+0000","last_unstale":"2024-09-05T13:12:25.169048+0000","last_undegraded":"2024-09-05T13:12:25.169048+0000","last_fullsized":"2024-09-05T13:12:25.169048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:03.155260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2623","reported_seq":2719,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:52.112926+0000","last_change":"2024-09-05T13:12:51.179418+0000","last_active":"2024-09-05T13:12:52.112926+0000","last_peered":"2024-09-05T13:12:52.112926+0000","last_clean":"2024-09-05T13:12:52.112926+0000","last_became_active":"2024-09-05T13:12:25.164969+0000","last_became_peered":"2024-09-05T13:12:25.164969+0000","last_unstale":"2024-09-05T13:12:52.112926+0000","last_undegraded":"2024-09-05T13:12:52.112926+0000","last_fullsized":"2024-09-05T13:12:52.112926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":2623,"log_dups_size":0,"ondisk_log_size":2623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:28:28.361737+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10719232,"num_objects":2617,"num_object_clones":0,"num_object_copies":5234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2613","reported_seq":2666,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.673721+0000","last_change":"2024-09-05T13:12:24.168530+0000","last_active":"2024-09-05T13:12:51.673721+0000","last_peered":"2024-09-05T13:12:51.673721+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:24.168300+0000","last_became_peered":"2024-09-05T13:12:24.168300+0000","last_unstale":"2024-09-05T13:12:51.673721+0000","last_undegraded":"2024-09-05T13:12:24.158160+0000","last_fullsized":"2024-09-05T13:12:51.673721+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2613,"log_dups_size":0,"ondisk_log_size":2613,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10695139,"num_objects":2612,"num_object_clones":0,"num_object_copies":5224,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":10445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2612}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162822+0000","last_change":"2024-09-05T13:12:24.163476+0000","last_active":"2024-09-05T13:12:25.162822+0000","last_peered":"2024-09-05T13:12:25.162822+0000","last_clean":"2024-09-05T13:12:25.162822+0000","last_became_active":"2024-09-05T13:12:24.163169+0000","last_became_peered":"2024-09-05T13:12:24.163169+0000","last_unstale":"2024-09-05T13:12:25.162822+0000","last_undegraded":"2024-09-05T13:12:25.162822+0000","last_fullsized":"2024-09-05T13:12:25.162822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:05:50.037488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2619","reported_seq":2674,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.682073+0000","last_change":"2024-09-05T13:12:24.168576+0000","last_active":"2024-09-05T13:12:51.682073+0000","last_peered":"2024-09-05T13:12:51.682073+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:24.168347+0000","last_became_peered":"2024-09-05T13:12:24.168347+0000","last_unstale":"2024-09-05T13:12:51.682073+0000","last_undegraded":"2024-09-05T13:12:24.157388+0000","last_fullsized":"2024-09-05T13:12:51.682073+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2619,"log_dups_size":0,"ondisk_log_size":2619,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10696105,"num_objects":2614,"num_object_clones":0,"num_object_copies":5228,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2616,"num_write_kb":10448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2614}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.166730+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:12:25.166730+0000","last_peered":"2024-09-05T13:12:25.166730+0000","last_clean":"2024-09-05T13:12:25.166730+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:12:25.166730+0000","last_undegraded":"2024-09-05T13:12:25.166730+0000","last_fullsized":"2024-09-05T13:12:25.166730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2680","reported_seq":2734,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:51.669971+0000","last_change":"2024-09-05T13:12:24.165983+0000","last_active":"2024-09-05T13:12:51.669971+0000","last_peered":"2024-09-05T13:12:51.669971+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:24.165544+0000","last_became_peered":"2024-09-05T13:12:24.165544+0000","last_unstale":"2024-09-05T13:12:51.669971+0000","last_undegraded":"2024-09-05T13:12:24.157734+0000","last_fullsized":"2024-09-05T13:12:51.669971+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2680,"log_dups_size":0,"ondisk_log_size":2680,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10961862,"num_objects":2678,"num_object_clones":0,"num_object_copies":5356,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2678,"num_write_kb":10706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2678}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162476+0000","last_change":"2024-09-05T13:12:24.165771+0000","last_active":"2024-09-05T13:12:25.162476+0000","last_peered":"2024-09-05T13:12:25.162476+0000","last_clean":"2024-09-05T13:12:25.162476+0000","last_became_active":"2024-09-05T13:12:24.165476+0000","last_became_peered":"2024-09-05T13:12:24.165476+0000","last_unstale":"2024-09-05T13:12:25.162476+0000","last_undegraded":"2024-09-05T13:12:25.162476+0000","last_fullsized":"2024-09-05T13:12:25.162476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:24.468878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.168816+0000","last_change":"2024-09-05T13:12:25.166728+0000","last_active":"2024-09-05T13:12:25.168816+0000","last_peered":"2024-09-05T13:12:25.168816+0000","last_clean":"2024-09-05T13:12:25.168816+0000","last_became_active":"2024-09-05T13:12:25.166434+0000","last_became_peered":"2024-09-05T13:12:25.166434+0000","last_unstale":"2024-09-05T13:12:25.168816+0000","last_undegraded":"2024-09-05T13:12:25.168816+0000","last_fullsized":"2024-09-05T13:12:25.168816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:26:25.045325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2660","reported_seq":2716,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:51.674293+0000","last_change":"2024-09-05T13:12:24.164240+0000","last_active":"2024-09-05T13:12:51.674293+0000","last_peered":"2024-09-05T13:12:51.674293+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:24.163127+0000","last_became_peered":"2024-09-05T13:12:24.163127+0000","last_unstale":"2024-09-05T13:12:51.674293+0000","last_undegraded":"2024-09-05T13:12:24.155686+0000","last_fullsized":"2024-09-05T13:12:51.674293+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2660,"log_dups_size":0,"ondisk_log_size":2660,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10848140,"num_objects":2652,"num_object_clones":0,"num_object_copies":5304,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2657,"num_write_kb":10599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":171996121,"num_objects":42008,"num_object_clones":0,"num_object_copies":84016,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30632,"num_write_kb":122414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344932352,"data_stored":344809510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42071,"ondisk_log_size":42071,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276624,"kb_used_data":164416,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94095216,"statfs":{"total":96636764160,"available":96353501184,"internally_reserved":0,"allocated":168361984,"data_stored":168153911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":114880173},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":272232,"kb_used_data":164248,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94099608,"statfs":{"total":96636764160,"available":96357998592,"internally_reserved":0,"allocated":168189952,"data_stored":167956124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110554794},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171151360,"data_stored":171129682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":173666304,"data_stored":173666304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:12:56.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 23 20 48642 48622 8.25707 6.05078 0.00509869 0.00945561 2024-09-05T13:12:57.187 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 24 20 50262 50242 8.17668 6.32812 0.0157345 0.00954817 2024-09-05T13:12:58.188 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 25 20 51843 51823 8.09648 6.17578 0.013487 0.00964071 2024-09-05T13:12:58.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:59.081 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:59.105 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-05T13:12:57.515719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":201499628,"num_objects":49213,"num_object_clones":0,"num_object_copies":98426,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37837,"num_write_kb":151228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49277,"ondisk_log_size":49277,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":608524,"kb_used_data":388332,"kb_used_omap":8,"kb_used_meta":220151,"kb_avail":188135156,"statfs":{"total":193273528320,"available":192650399744,"internally_reserved":0,"allocated":397651968,"data_stored":397210067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":225434967},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,3,7,7,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":103542533,"num_objects":25271,"num_object_clones":0,"num_object_copies":50542,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":25271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13868,"num_write_kb":55532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002681"},"pg_stats":[{"pgid":"2.f","version":"36'2956","reported_seq":3007,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:56.124336+0000","last_change":"2024-09-05T13:12:24.168600+0000","last_active":"2024-09-05T13:12:56.124336+0000","last_peered":"2024-09-05T13:12:56.124336+0000","last_clean":"2024-09-05T13:12:56.124336+0000","last_became_active":"2024-09-05T13:12:24.168356+0000","last_became_peered":"2024-09-05T13:12:24.168356+0000","last_unstale":"2024-09-05T13:12:56.124336+0000","last_undegraded":"2024-09-05T13:12:56.124336+0000","last_fullsized":"2024-09-05T13:12:56.124336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2956,"log_dups_size":0,"ondisk_log_size":2956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:44.376826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12107776,"num_objects":2956,"num_object_clones":0,"num_object_copies":5912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2956,"num_write_kb":11824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'3142","reported_seq":3227,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.139567+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:12:57.139567+0000","last_peered":"2024-09-05T13:12:57.139567+0000","last_clean":"2024-09-05T13:12:57.139567+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:12:57.139567+0000","last_undegraded":"2024-09-05T13:12:57.139567+0000","last_fullsized":"2024-09-05T13:12:57.139567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":3142,"log_dups_size":0,"ondisk_log_size":3142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845056,"num_objects":3136,"num_object_clones":0,"num_object_copies":6272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1152,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'3210","reported_seq":3312,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.146397+0000","last_change":"2024-09-05T13:12:47.044143+0000","last_active":"2024-09-05T13:12:57.146397+0000","last_peered":"2024-09-05T13:12:57.146397+0000","last_clean":"2024-09-05T13:12:57.146397+0000","last_became_active":"2024-09-05T13:12:25.166237+0000","last_became_peered":"2024-09-05T13:12:25.166237+0000","last_unstale":"2024-09-05T13:12:57.146397+0000","last_undegraded":"2024-09-05T13:12:57.146397+0000","last_fullsized":"2024-09-05T13:12:57.146397+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":3210,"log_dups_size":0,"ondisk_log_size":3210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:32:20.708885+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13131776,"num_objects":3206,"num_object_clones":0,"num_object_copies":6412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1021,"num_write_kb":4084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'3084","reported_seq":3134,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.134191+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:12:57.134191+0000","last_peered":"2024-09-05T13:12:57.134191+0000","last_clean":"2024-09-05T13:12:57.134191+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:12:57.134191+0000","last_undegraded":"2024-09-05T13:12:57.134191+0000","last_fullsized":"2024-09-05T13:12:57.134191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3084,"log_dups_size":0,"ondisk_log_size":3084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12632064,"num_objects":3084,"num_object_clones":0,"num_object_copies":6168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3084,"num_write_kb":12336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'3066","reported_seq":3125,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.119175+0000","last_change":"2024-09-05T13:12:24.165681+0000","last_active":"2024-09-05T13:12:56.119175+0000","last_peered":"2024-09-05T13:12:56.119175+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:12:56.119175+0000","last_undegraded":"2024-09-05T13:12:24.156268+0000","last_fullsized":"2024-09-05T13:12:56.119175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3066,"log_dups_size":0,"ondisk_log_size":3066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12530630,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3064,"num_write_kb":12240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'3038","reported_seq":3094,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.134904+0000","last_change":"2024-09-05T13:12:24.165709+0000","last_active":"2024-09-05T13:12:56.134904+0000","last_peered":"2024-09-05T13:12:56.134904+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:24.165469+0000","last_became_peered":"2024-09-05T13:12:24.165469+0000","last_unstale":"2024-09-05T13:12:56.134904+0000","last_undegraded":"2024-09-05T13:12:24.157014+0000","last_fullsized":"2024-09-05T13:12:56.134904+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3038,"log_dups_size":0,"ondisk_log_size":3038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12424134,"num_objects":3035,"num_object_clones":0,"num_object_copies":6070,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3036,"num_write_kb":12135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3035}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'3044","reported_seq":3101,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.121689+0000","last_change":"2024-09-05T13:12:24.165854+0000","last_active":"2024-09-05T13:12:56.121689+0000","last_peered":"2024-09-05T13:12:56.121689+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:24.165604+0000","last_became_peered":"2024-09-05T13:12:24.165604+0000","last_unstale":"2024-09-05T13:12:56.121689+0000","last_undegraded":"2024-09-05T13:12:24.157554+0000","last_fullsized":"2024-09-05T13:12:56.121689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3044,"log_dups_size":0,"ondisk_log_size":3044,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12444614,"num_objects":3040,"num_object_clones":0,"num_object_copies":6080,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3040}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'3062","reported_seq":3123,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.119689+0000","last_change":"2024-09-05T13:12:24.163654+0000","last_active":"2024-09-05T13:12:56.119689+0000","last_peered":"2024-09-05T13:12:56.119689+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:12:56.119689+0000","last_undegraded":"2024-09-05T13:12:24.156202+0000","last_fullsized":"2024-09-05T13:12:56.119689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3062,"log_dups_size":0,"ondisk_log_size":3062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534243,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3061,"num_write_kb":12241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.160861+0000","last_change":"2024-09-05T13:12:24.163255+0000","last_active":"2024-09-05T13:12:25.160861+0000","last_peered":"2024-09-05T13:12:25.160861+0000","last_clean":"2024-09-05T13:12:25.160861+0000","last_became_active":"2024-09-05T13:12:24.162958+0000","last_became_peered":"2024-09-05T13:12:24.162958+0000","last_unstale":"2024-09-05T13:12:25.160861+0000","last_undegraded":"2024-09-05T13:12:25.160861+0000","last_fullsized":"2024-09-05T13:12:25.160861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:58.405279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'3151","reported_seq":3256,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.138889+0000","last_change":"2024-09-05T13:12:49.050077+0000","last_active":"2024-09-05T13:12:57.138889+0000","last_peered":"2024-09-05T13:12:57.138889+0000","last_clean":"2024-09-05T13:12:57.138889+0000","last_became_active":"2024-09-05T13:12:25.167319+0000","last_became_peered":"2024-09-05T13:12:25.167319+0000","last_unstale":"2024-09-05T13:12:57.138889+0000","last_undegraded":"2024-09-05T13:12:57.138889+0000","last_fullsized":"2024-09-05T13:12:57.138889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":3151,"log_dups_size":0,"ondisk_log_size":3151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:33.891462+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12877824,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":805,"num_write_kb":3220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.165344+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:12:25.165344+0000","last_peered":"2024-09-05T13:12:25.165344+0000","last_clean":"2024-09-05T13:12:25.165344+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:12:25.165344+0000","last_undegraded":"2024-09-05T13:12:25.165344+0000","last_fullsized":"2024-09-05T13:12:25.165344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'3040","reported_seq":3090,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.133157+0000","last_change":"2024-09-05T13:12:24.165760+0000","last_active":"2024-09-05T13:12:56.133157+0000","last_peered":"2024-09-05T13:12:56.133157+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:24.165511+0000","last_became_peered":"2024-09-05T13:12:24.165511+0000","last_unstale":"2024-09-05T13:12:56.133157+0000","last_undegraded":"2024-09-05T13:12:24.157434+0000","last_fullsized":"2024-09-05T13:12:56.133157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3040,"log_dups_size":0,"ondisk_log_size":3040,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12420038,"num_objects":3034,"num_object_clones":0,"num_object_copies":6068,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3038,"num_write_kb":12132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3034}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162067+0000","last_change":"2024-09-05T13:12:24.164231+0000","last_active":"2024-09-05T13:12:25.162067+0000","last_peered":"2024-09-05T13:12:25.162067+0000","last_clean":"2024-09-05T13:12:03.089565+0000","last_became_active":"2024-09-05T13:12:24.163048+0000","last_became_peered":"2024-09-05T13:12:24.163048+0000","last_unstale":"2024-09-05T13:12:25.162067+0000","last_undegraded":"2024-09-05T13:12:24.156585+0000","last_fullsized":"2024-09-05T13:12:25.162067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'3099","reported_seq":3192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.151915+0000","last_change":"2024-09-05T13:12:50.064282+0000","last_active":"2024-09-05T13:12:57.151915+0000","last_peered":"2024-09-05T13:12:57.151915+0000","last_clean":"2024-09-05T13:12:57.151915+0000","last_became_active":"2024-09-05T13:12:25.165724+0000","last_became_peered":"2024-09-05T13:12:25.165724+0000","last_unstale":"2024-09-05T13:12:57.151915+0000","last_undegraded":"2024-09-05T13:12:57.151915+0000","last_fullsized":"2024-09-05T13:12:57.151915+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":3099,"log_dups_size":0,"ondisk_log_size":3099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:47:05.084065+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12673024,"num_objects":3094,"num_object_clones":0,"num_object_copies":6188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.169048+0000","last_change":"2024-09-05T13:12:25.167242+0000","last_active":"2024-09-05T13:12:25.169048+0000","last_peered":"2024-09-05T13:12:25.169048+0000","last_clean":"2024-09-05T13:12:25.169048+0000","last_became_active":"2024-09-05T13:12:25.165964+0000","last_became_peered":"2024-09-05T13:12:25.165964+0000","last_unstale":"2024-09-05T13:12:25.169048+0000","last_undegraded":"2024-09-05T13:12:25.169048+0000","last_fullsized":"2024-09-05T13:12:25.169048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:03.155260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'3130","reported_seq":3226,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.099937+0000","last_change":"2024-09-05T13:12:51.179418+0000","last_active":"2024-09-05T13:12:57.099937+0000","last_peered":"2024-09-05T13:12:57.099937+0000","last_clean":"2024-09-05T13:12:57.099937+0000","last_became_active":"2024-09-05T13:12:25.164969+0000","last_became_peered":"2024-09-05T13:12:25.164969+0000","last_unstale":"2024-09-05T13:12:57.099937+0000","last_undegraded":"2024-09-05T13:12:57.099937+0000","last_fullsized":"2024-09-05T13:12:57.099937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":3130,"log_dups_size":0,"ondisk_log_size":3130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:28:28.361737+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12795904,"num_objects":3124,"num_object_clones":0,"num_object_copies":6248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":639,"num_write_kb":2556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'3034","reported_seq":3090,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.124417+0000","last_change":"2024-09-05T13:12:24.168530+0000","last_active":"2024-09-05T13:12:56.124417+0000","last_peered":"2024-09-05T13:12:56.124417+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:24.168300+0000","last_became_peered":"2024-09-05T13:12:24.168300+0000","last_unstale":"2024-09-05T13:12:56.124417+0000","last_undegraded":"2024-09-05T13:12:24.158160+0000","last_fullsized":"2024-09-05T13:12:56.124417+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3034,"log_dups_size":0,"ondisk_log_size":3034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419555,"num_objects":3033,"num_object_clones":0,"num_object_copies":6066,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3033}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162822+0000","last_change":"2024-09-05T13:12:24.163476+0000","last_active":"2024-09-05T13:12:25.162822+0000","last_peered":"2024-09-05T13:12:25.162822+0000","last_clean":"2024-09-05T13:12:25.162822+0000","last_became_active":"2024-09-05T13:12:24.163169+0000","last_became_peered":"2024-09-05T13:12:24.163169+0000","last_unstale":"2024-09-05T13:12:25.162822+0000","last_undegraded":"2024-09-05T13:12:25.162822+0000","last_fullsized":"2024-09-05T13:12:25.162822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:05:50.037488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'3036","reported_seq":3091,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.100266+0000","last_change":"2024-09-05T13:12:24.168576+0000","last_active":"2024-09-05T13:12:56.100266+0000","last_peered":"2024-09-05T13:12:56.100266+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:24.168347+0000","last_became_peered":"2024-09-05T13:12:24.168347+0000","last_unstale":"2024-09-05T13:12:56.100266+0000","last_undegraded":"2024-09-05T13:12:24.157388+0000","last_fullsized":"2024-09-05T13:12:56.100266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3036,"log_dups_size":0,"ondisk_log_size":3036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12404137,"num_objects":3031,"num_object_clones":0,"num_object_copies":6062,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":12116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3031}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.166730+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:12:25.166730+0000","last_peered":"2024-09-05T13:12:25.166730+0000","last_clean":"2024-09-05T13:12:25.166730+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:12:25.166730+0000","last_undegraded":"2024-09-05T13:12:25.166730+0000","last_fullsized":"2024-09-05T13:12:25.166730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'3109","reported_seq":3162,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.139017+0000","last_change":"2024-09-05T13:12:24.165983+0000","last_active":"2024-09-05T13:12:56.139017+0000","last_peered":"2024-09-05T13:12:56.139017+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:24.165544+0000","last_became_peered":"2024-09-05T13:12:24.165544+0000","last_unstale":"2024-09-05T13:12:56.139017+0000","last_undegraded":"2024-09-05T13:12:24.157734+0000","last_fullsized":"2024-09-05T13:12:56.139017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3109,"log_dups_size":0,"ondisk_log_size":3109,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12719046,"num_objects":3107,"num_object_clones":0,"num_object_copies":6214,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3107,"num_write_kb":12422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3107}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162476+0000","last_change":"2024-09-05T13:12:24.165771+0000","last_active":"2024-09-05T13:12:25.162476+0000","last_peered":"2024-09-05T13:12:25.162476+0000","last_clean":"2024-09-05T13:12:25.162476+0000","last_became_active":"2024-09-05T13:12:24.165476+0000","last_became_peered":"2024-09-05T13:12:24.165476+0000","last_unstale":"2024-09-05T13:12:25.162476+0000","last_undegraded":"2024-09-05T13:12:25.162476+0000","last_fullsized":"2024-09-05T13:12:25.162476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:24.468878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.168816+0000","last_change":"2024-09-05T13:12:25.166728+0000","last_active":"2024-09-05T13:12:25.168816+0000","last_peered":"2024-09-05T13:12:25.168816+0000","last_clean":"2024-09-05T13:12:25.168816+0000","last_became_active":"2024-09-05T13:12:25.166434+0000","last_became_peered":"2024-09-05T13:12:25.166434+0000","last_unstale":"2024-09-05T13:12:25.168816+0000","last_undegraded":"2024-09-05T13:12:25.168816+0000","last_fullsized":"2024-09-05T13:12:25.168816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:26:25.045325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'3073","reported_seq":3129,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:56.118970+0000","last_change":"2024-09-05T13:12:24.164240+0000","last_active":"2024-09-05T13:12:56.118970+0000","last_peered":"2024-09-05T13:12:56.118970+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:24.163127+0000","last_became_peered":"2024-09-05T13:12:24.163127+0000","last_unstale":"2024-09-05T13:12:56.118970+0000","last_undegraded":"2024-09-05T13:12:24.155686+0000","last_fullsized":"2024-09-05T13:12:56.118970+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3073,"log_dups_size":0,"ondisk_log_size":3073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12539788,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":12251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3064}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":201499609,"num_objects":49211,"num_object_clones":0,"num_object_copies":98422,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37835,"num_write_kb":151226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":404697088,"data_stored":404570633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49274,"ondisk_log_size":49274,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306724,"kb_used_data":194516,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94065116,"statfs":{"total":96636764160,"available":96322678784,"internally_reserved":0,"allocated":199184384,"data_stored":198976311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":114880173},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301800,"kb_used_data":193816,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94070040,"statfs":{"total":96636764160,"available":96327720960,"internally_reserved":0,"allocated":198467584,"data_stored":198233756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110554794},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,4,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":198922240,"data_stored":198900562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205660160,"data_stored":205656547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:12:59.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:12:59.188 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 26 20 53096 53076 7.97332 4.89453 0.0115055 0.00979094 2024-09-05T13:12:59.641 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:12:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:12:59.668 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-05T13:12:59.516157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":201499628,"num_objects":49213,"num_object_clones":0,"num_object_copies":98426,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37837,"num_write_kb":151228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49277,"ondisk_log_size":49277,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":608524,"kb_used_data":388332,"kb_used_omap":8,"kb_used_meta":220151,"kb_avail":188135156,"statfs":{"total":193273528320,"available":192650399744,"internally_reserved":0,"allocated":397651968,"data_stored":397210067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":225434967},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,3,7,7,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":63603886,"num_objects":15523,"num_object_clones":0,"num_object_copies":31046,"num_objects_missing_on_primary":-6,"num_objects_missing":-6,"num_objects_degraded":-12,"num_objects_misplaced":0,"num_objects_unfound":-20,"num_objects_dirty":15523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8293,"num_write_kb":33210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002669"},"pg_stats":[{"pgid":"2.f","version":"36'2956","reported_seq":3007,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:56.124336+0000","last_change":"2024-09-05T13:12:24.168600+0000","last_active":"2024-09-05T13:12:56.124336+0000","last_peered":"2024-09-05T13:12:56.124336+0000","last_clean":"2024-09-05T13:12:56.124336+0000","last_became_active":"2024-09-05T13:12:24.168356+0000","last_became_peered":"2024-09-05T13:12:24.168356+0000","last_unstale":"2024-09-05T13:12:56.124336+0000","last_undegraded":"2024-09-05T13:12:56.124336+0000","last_fullsized":"2024-09-05T13:12:56.124336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":2956,"log_dups_size":0,"ondisk_log_size":2956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:44.376826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12107776,"num_objects":2956,"num_object_clones":0,"num_object_copies":5912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2956,"num_write_kb":11824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'3142","reported_seq":3227,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.139567+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:12:57.139567+0000","last_peered":"2024-09-05T13:12:57.139567+0000","last_clean":"2024-09-05T13:12:57.139567+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:12:57.139567+0000","last_undegraded":"2024-09-05T13:12:57.139567+0000","last_fullsized":"2024-09-05T13:12:57.139567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":3142,"log_dups_size":0,"ondisk_log_size":3142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845056,"num_objects":3136,"num_object_clones":0,"num_object_copies":6272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1152,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'3210","reported_seq":3312,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.146397+0000","last_change":"2024-09-05T13:12:47.044143+0000","last_active":"2024-09-05T13:12:57.146397+0000","last_peered":"2024-09-05T13:12:57.146397+0000","last_clean":"2024-09-05T13:12:57.146397+0000","last_became_active":"2024-09-05T13:12:25.166237+0000","last_became_peered":"2024-09-05T13:12:25.166237+0000","last_unstale":"2024-09-05T13:12:57.146397+0000","last_undegraded":"2024-09-05T13:12:57.146397+0000","last_fullsized":"2024-09-05T13:12:57.146397+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":3210,"log_dups_size":0,"ondisk_log_size":3210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:32:20.708885+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13131776,"num_objects":3206,"num_object_clones":0,"num_object_copies":6412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1021,"num_write_kb":4084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'3084","reported_seq":3134,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.134191+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:12:57.134191+0000","last_peered":"2024-09-05T13:12:57.134191+0000","last_clean":"2024-09-05T13:12:57.134191+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:12:57.134191+0000","last_undegraded":"2024-09-05T13:12:57.134191+0000","last_fullsized":"2024-09-05T13:12:57.134191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3084,"log_dups_size":0,"ondisk_log_size":3084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12632064,"num_objects":3084,"num_object_clones":0,"num_object_copies":6168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3084,"num_write_kb":12336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'3066","reported_seq":3125,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.119175+0000","last_change":"2024-09-05T13:12:24.165681+0000","last_active":"2024-09-05T13:12:56.119175+0000","last_peered":"2024-09-05T13:12:56.119175+0000","last_clean":"2024-09-05T13:11:51.768830+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:12:56.119175+0000","last_undegraded":"2024-09-05T13:12:24.156268+0000","last_fullsized":"2024-09-05T13:12:56.119175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3066,"log_dups_size":0,"ondisk_log_size":3066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12530630,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3064,"num_write_kb":12240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'3038","reported_seq":3094,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.134904+0000","last_change":"2024-09-05T13:12:24.165709+0000","last_active":"2024-09-05T13:12:56.134904+0000","last_peered":"2024-09-05T13:12:56.134904+0000","last_clean":"2024-09-05T13:11:52.582751+0000","last_became_active":"2024-09-05T13:12:24.165469+0000","last_became_peered":"2024-09-05T13:12:24.165469+0000","last_unstale":"2024-09-05T13:12:56.134904+0000","last_undegraded":"2024-09-05T13:12:24.157014+0000","last_fullsized":"2024-09-05T13:12:56.134904+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3038,"log_dups_size":0,"ondisk_log_size":3038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12424134,"num_objects":3035,"num_object_clones":0,"num_object_copies":6070,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3036,"num_write_kb":12135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3035}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'3044","reported_seq":3101,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.121689+0000","last_change":"2024-09-05T13:12:24.165854+0000","last_active":"2024-09-05T13:12:56.121689+0000","last_peered":"2024-09-05T13:12:56.121689+0000","last_clean":"2024-09-05T13:11:52.008925+0000","last_became_active":"2024-09-05T13:12:24.165604+0000","last_became_peered":"2024-09-05T13:12:24.165604+0000","last_unstale":"2024-09-05T13:12:56.121689+0000","last_undegraded":"2024-09-05T13:12:24.157554+0000","last_fullsized":"2024-09-05T13:12:56.121689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3044,"log_dups_size":0,"ondisk_log_size":3044,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12444614,"num_objects":3040,"num_object_clones":0,"num_object_copies":6080,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3040}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'3062","reported_seq":3123,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.119689+0000","last_change":"2024-09-05T13:12:24.163654+0000","last_active":"2024-09-05T13:12:56.119689+0000","last_peered":"2024-09-05T13:12:56.119689+0000","last_clean":"2024-09-05T13:11:41.951561+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:12:56.119689+0000","last_undegraded":"2024-09-05T13:12:24.156202+0000","last_fullsized":"2024-09-05T13:12:56.119689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3062,"log_dups_size":0,"ondisk_log_size":3062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534243,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3061,"num_write_kb":12241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.160861+0000","last_change":"2024-09-05T13:12:24.163255+0000","last_active":"2024-09-05T13:12:25.160861+0000","last_peered":"2024-09-05T13:12:25.160861+0000","last_clean":"2024-09-05T13:12:25.160861+0000","last_became_active":"2024-09-05T13:12:24.162958+0000","last_became_peered":"2024-09-05T13:12:24.162958+0000","last_unstale":"2024-09-05T13:12:25.160861+0000","last_undegraded":"2024-09-05T13:12:25.160861+0000","last_fullsized":"2024-09-05T13:12:25.160861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:58.405279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'3151","reported_seq":3256,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.138889+0000","last_change":"2024-09-05T13:12:49.050077+0000","last_active":"2024-09-05T13:12:57.138889+0000","last_peered":"2024-09-05T13:12:57.138889+0000","last_clean":"2024-09-05T13:12:57.138889+0000","last_became_active":"2024-09-05T13:12:25.167319+0000","last_became_peered":"2024-09-05T13:12:25.167319+0000","last_unstale":"2024-09-05T13:12:57.138889+0000","last_undegraded":"2024-09-05T13:12:57.138889+0000","last_fullsized":"2024-09-05T13:12:57.138889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":3151,"log_dups_size":0,"ondisk_log_size":3151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:33.891462+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12877824,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":805,"num_write_kb":3220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.165344+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:12:25.165344+0000","last_peered":"2024-09-05T13:12:25.165344+0000","last_clean":"2024-09-05T13:12:25.165344+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:12:25.165344+0000","last_undegraded":"2024-09-05T13:12:25.165344+0000","last_fullsized":"2024-09-05T13:12:25.165344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'3040","reported_seq":3090,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.133157+0000","last_change":"2024-09-05T13:12:24.165760+0000","last_active":"2024-09-05T13:12:56.133157+0000","last_peered":"2024-09-05T13:12:56.133157+0000","last_clean":"2024-09-05T13:11:52.632293+0000","last_became_active":"2024-09-05T13:12:24.165511+0000","last_became_peered":"2024-09-05T13:12:24.165511+0000","last_unstale":"2024-09-05T13:12:56.133157+0000","last_undegraded":"2024-09-05T13:12:24.157434+0000","last_fullsized":"2024-09-05T13:12:56.133157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3040,"log_dups_size":0,"ondisk_log_size":3040,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12420038,"num_objects":3034,"num_object_clones":0,"num_object_copies":6068,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3038,"num_write_kb":12132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3034}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:25.162067+0000","last_change":"2024-09-05T13:12:24.164231+0000","last_active":"2024-09-05T13:12:25.162067+0000","last_peered":"2024-09-05T13:12:25.162067+0000","last_clean":"2024-09-05T13:12:03.089565+0000","last_became_active":"2024-09-05T13:12:24.163048+0000","last_became_peered":"2024-09-05T13:12:24.163048+0000","last_unstale":"2024-09-05T13:12:25.162067+0000","last_undegraded":"2024-09-05T13:12:24.156585+0000","last_fullsized":"2024-09-05T13:12:25.162067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'3099","reported_seq":3192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.151915+0000","last_change":"2024-09-05T13:12:50.064282+0000","last_active":"2024-09-05T13:12:57.151915+0000","last_peered":"2024-09-05T13:12:57.151915+0000","last_clean":"2024-09-05T13:12:57.151915+0000","last_became_active":"2024-09-05T13:12:25.165724+0000","last_became_peered":"2024-09-05T13:12:25.165724+0000","last_unstale":"2024-09-05T13:12:57.151915+0000","last_undegraded":"2024-09-05T13:12:57.151915+0000","last_fullsized":"2024-09-05T13:12:57.151915+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":3099,"log_dups_size":0,"ondisk_log_size":3099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:47:05.084065+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12673024,"num_objects":3094,"num_object_clones":0,"num_object_copies":6188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.169048+0000","last_change":"2024-09-05T13:12:25.167242+0000","last_active":"2024-09-05T13:12:25.169048+0000","last_peered":"2024-09-05T13:12:25.169048+0000","last_clean":"2024-09-05T13:12:25.169048+0000","last_became_active":"2024-09-05T13:12:25.165964+0000","last_became_peered":"2024-09-05T13:12:25.165964+0000","last_unstale":"2024-09-05T13:12:25.169048+0000","last_undegraded":"2024-09-05T13:12:25.169048+0000","last_fullsized":"2024-09-05T13:12:25.169048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:03.155260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'3130","reported_seq":3226,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.099937+0000","last_change":"2024-09-05T13:12:51.179418+0000","last_active":"2024-09-05T13:12:57.099937+0000","last_peered":"2024-09-05T13:12:57.099937+0000","last_clean":"2024-09-05T13:12:57.099937+0000","last_became_active":"2024-09-05T13:12:25.164969+0000","last_became_peered":"2024-09-05T13:12:25.164969+0000","last_unstale":"2024-09-05T13:12:57.099937+0000","last_undegraded":"2024-09-05T13:12:57.099937+0000","last_fullsized":"2024-09-05T13:12:57.099937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":3130,"log_dups_size":0,"ondisk_log_size":3130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:28:28.361737+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12795904,"num_objects":3124,"num_object_clones":0,"num_object_copies":6248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":639,"num_write_kb":2556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'3034","reported_seq":3090,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.124417+0000","last_change":"2024-09-05T13:12:24.168530+0000","last_active":"2024-09-05T13:12:56.124417+0000","last_peered":"2024-09-05T13:12:56.124417+0000","last_clean":"2024-09-05T13:11:44.973255+0000","last_became_active":"2024-09-05T13:12:24.168300+0000","last_became_peered":"2024-09-05T13:12:24.168300+0000","last_unstale":"2024-09-05T13:12:56.124417+0000","last_undegraded":"2024-09-05T13:12:24.158160+0000","last_fullsized":"2024-09-05T13:12:56.124417+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3034,"log_dups_size":0,"ondisk_log_size":3034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419555,"num_objects":3033,"num_object_clones":0,"num_object_copies":6066,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":12129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3033}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162822+0000","last_change":"2024-09-05T13:12:24.163476+0000","last_active":"2024-09-05T13:12:25.162822+0000","last_peered":"2024-09-05T13:12:25.162822+0000","last_clean":"2024-09-05T13:12:25.162822+0000","last_became_active":"2024-09-05T13:12:24.163169+0000","last_became_peered":"2024-09-05T13:12:24.163169+0000","last_unstale":"2024-09-05T13:12:25.162822+0000","last_undegraded":"2024-09-05T13:12:25.162822+0000","last_fullsized":"2024-09-05T13:12:25.162822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:05:50.037488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'3036","reported_seq":3091,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.100266+0000","last_change":"2024-09-05T13:12:24.168576+0000","last_active":"2024-09-05T13:12:56.100266+0000","last_peered":"2024-09-05T13:12:56.100266+0000","last_clean":"2024-09-05T13:11:52.535201+0000","last_became_active":"2024-09-05T13:12:24.168347+0000","last_became_peered":"2024-09-05T13:12:24.168347+0000","last_unstale":"2024-09-05T13:12:56.100266+0000","last_undegraded":"2024-09-05T13:12:24.157388+0000","last_fullsized":"2024-09-05T13:12:56.100266+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3036,"log_dups_size":0,"ondisk_log_size":3036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12404137,"num_objects":3031,"num_object_clones":0,"num_object_copies":6062,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":12116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3031}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.166730+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:12:25.166730+0000","last_peered":"2024-09-05T13:12:25.166730+0000","last_clean":"2024-09-05T13:12:25.166730+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:12:25.166730+0000","last_undegraded":"2024-09-05T13:12:25.166730+0000","last_fullsized":"2024-09-05T13:12:25.166730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'3109","reported_seq":3162,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-05T13:12:56.139017+0000","last_change":"2024-09-05T13:12:24.165983+0000","last_active":"2024-09-05T13:12:56.139017+0000","last_peered":"2024-09-05T13:12:56.139017+0000","last_clean":"2024-09-05T13:11:44.973336+0000","last_became_active":"2024-09-05T13:12:24.165544+0000","last_became_peered":"2024-09-05T13:12:24.165544+0000","last_unstale":"2024-09-05T13:12:56.139017+0000","last_undegraded":"2024-09-05T13:12:24.157734+0000","last_fullsized":"2024-09-05T13:12:56.139017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3109,"log_dups_size":0,"ondisk_log_size":3109,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12719046,"num_objects":3107,"num_object_clones":0,"num_object_copies":6214,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3107,"num_write_kb":12422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3107}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162476+0000","last_change":"2024-09-05T13:12:24.165771+0000","last_active":"2024-09-05T13:12:25.162476+0000","last_peered":"2024-09-05T13:12:25.162476+0000","last_clean":"2024-09-05T13:12:25.162476+0000","last_became_active":"2024-09-05T13:12:24.165476+0000","last_became_peered":"2024-09-05T13:12:24.165476+0000","last_unstale":"2024-09-05T13:12:25.162476+0000","last_undegraded":"2024-09-05T13:12:25.162476+0000","last_fullsized":"2024-09-05T13:12:25.162476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:24.468878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.168816+0000","last_change":"2024-09-05T13:12:25.166728+0000","last_active":"2024-09-05T13:12:25.168816+0000","last_peered":"2024-09-05T13:12:25.168816+0000","last_clean":"2024-09-05T13:12:25.168816+0000","last_became_active":"2024-09-05T13:12:25.166434+0000","last_became_peered":"2024-09-05T13:12:25.166434+0000","last_unstale":"2024-09-05T13:12:25.168816+0000","last_undegraded":"2024-09-05T13:12:25.168816+0000","last_fullsized":"2024-09-05T13:12:25.168816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:26:25.045325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'3073","reported_seq":3129,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-05T13:12:56.118970+0000","last_change":"2024-09-05T13:12:24.164240+0000","last_active":"2024-09-05T13:12:56.118970+0000","last_peered":"2024-09-05T13:12:56.118970+0000","last_clean":"2024-09-05T13:11:52.297423+0000","last_became_active":"2024-09-05T13:12:24.163127+0000","last_became_peered":"2024-09-05T13:12:24.163127+0000","last_unstale":"2024-09-05T13:12:56.118970+0000","last_undegraded":"2024-09-05T13:12:24.155686+0000","last_fullsized":"2024-09-05T13:12:56.118970+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3073,"log_dups_size":0,"ondisk_log_size":3073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12539788,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":12251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3064}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":201499609,"num_objects":49211,"num_object_clones":0,"num_object_copies":98422,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37835,"num_write_kb":151226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":404697088,"data_stored":404570633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49274,"ondisk_log_size":49274,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306724,"kb_used_data":194516,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94065116,"statfs":{"total":96636764160,"available":96322678784,"internally_reserved":0,"allocated":199184384,"data_stored":198976311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":114880173},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301800,"kb_used_data":193816,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94070040,"statfs":{"total":96636764160,"available":96327720960,"internally_reserved":0,"allocated":198467584,"data_stored":198233756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110554794},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,4,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":198922240,"data_stored":198900562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205660160,"data_stored":205656547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:00.188 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 27 20 54402 54382 7.86695 5.10156 0.0114036 0.00992297 2024-09-05T13:13:01.188 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 28 20 55804 55784 7.78156 5.47656 0.0112338 0.0100332 2024-09-05T13:13:02.189 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 29 20 57165 57145 7.6961 5.31641 0.0102317 0.0101452 2024-09-05T13:13:02.670 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:03.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 30 20 58420 58400 7.60297 4.90234 0.0148341 0.0102679 2024-09-05T13:13:03.221 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:03.224 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:03.242 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-05T13:13:01.516515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":219880544,"num_objects":53697,"num_object_clones":0,"num_object_copies":107394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35607,"num_write_kb":142340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53765,"ondisk_log_size":53765,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":660072,"kb_used_data":417672,"kb_used_omap":8,"kb_used_meta":242359,"kb_avail":188083608,"statfs":{"total":193273528320,"available":192597614592,"internally_reserved":0,"allocated":427696128,"data_stored":427272292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8873,"internal_metadata":248175959},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,5,10,10,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":81984802,"num_objects":20007,"num_object_clones":0,"num_object_copies":40014,"num_objects_missing_on_primary":-24,"num_objects_missing":-24,"num_objects_degraded":-44,"num_objects_misplaced":0,"num_objects_unfound":-20,"num_objects_dirty":20007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6063,"num_write_kb":24322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002527"},"pg_stats":[{"pgid":"2.f","version":"36'3411","reported_seq":3461,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:13:01.112362+0000","last_change":"2024-09-05T13:12:24.168600+0000","last_active":"2024-09-05T13:13:01.112362+0000","last_peered":"2024-09-05T13:13:01.112362+0000","last_clean":"2024-09-05T13:13:01.112362+0000","last_became_active":"2024-09-05T13:12:24.168356+0000","last_became_peered":"2024-09-05T13:12:24.168356+0000","last_unstale":"2024-09-05T13:13:01.112362+0000","last_undegraded":"2024-09-05T13:13:01.112362+0000","last_fullsized":"2024-09-05T13:13:01.112362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3411,"log_dups_size":0,"ondisk_log_size":3411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:44.376826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13971456,"num_objects":3411,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3411,"num_write_kb":13644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'3142","reported_seq":3227,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.139567+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:12:57.139567+0000","last_peered":"2024-09-05T13:12:57.139567+0000","last_clean":"2024-09-05T13:12:57.139567+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:12:57.139567+0000","last_undegraded":"2024-09-05T13:12:57.139567+0000","last_fullsized":"2024-09-05T13:12:57.139567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":3142,"log_dups_size":0,"ondisk_log_size":3142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845056,"num_objects":3136,"num_object_clones":0,"num_object_copies":6272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1152,"num_write_kb":4608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'3210","reported_seq":3312,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.146397+0000","last_change":"2024-09-05T13:12:47.044143+0000","last_active":"2024-09-05T13:12:57.146397+0000","last_peered":"2024-09-05T13:12:57.146397+0000","last_clean":"2024-09-05T13:12:57.146397+0000","last_became_active":"2024-09-05T13:12:25.166237+0000","last_became_peered":"2024-09-05T13:12:25.166237+0000","last_unstale":"2024-09-05T13:12:57.146397+0000","last_undegraded":"2024-09-05T13:12:57.146397+0000","last_fullsized":"2024-09-05T13:12:57.146397+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":3210,"log_dups_size":0,"ondisk_log_size":3210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:32:20.708885+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13131776,"num_objects":3206,"num_object_clones":0,"num_object_copies":6412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1021,"num_write_kb":4084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'3084","reported_seq":3134,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.134191+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:12:57.134191+0000","last_peered":"2024-09-05T13:12:57.134191+0000","last_clean":"2024-09-05T13:12:57.134191+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:12:57.134191+0000","last_undegraded":"2024-09-05T13:12:57.134191+0000","last_fullsized":"2024-09-05T13:12:57.134191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3084,"log_dups_size":0,"ondisk_log_size":3084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12632064,"num_objects":3084,"num_object_clones":0,"num_object_copies":6168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3084,"num_write_kb":12336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'3497","reported_seq":3592,"reported_epoch":36,"state":"active+clean+scrubbing","last_fresh":"2024-09-05T13:13:01.144931+0000","last_change":"2024-09-05T13:13:00.084318+0000","last_active":"2024-09-05T13:13:01.144931+0000","last_peered":"2024-09-05T13:13:01.144931+0000","last_clean":"2024-09-05T13:13:01.144931+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:01.144931+0000","last_undegraded":"2024-09-05T13:13:01.144931+0000","last_fullsized":"2024-09-05T13:13:01.144931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":2574,"log_size":3497,"log_dups_size":0,"ondisk_log_size":3497,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14296006,"num_objects":3492,"num_object_clones":0,"num_object_copies":6984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3495,"num_write_kb":13964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'3487","reported_seq":3559,"reported_epoch":36,"state":"active+clean+scrubbing","last_fresh":"2024-09-05T13:13:01.168424+0000","last_change":"2024-09-05T13:13:01.123064+0000","last_active":"2024-09-05T13:13:01.168424+0000","last_peered":"2024-09-05T13:13:01.168424+0000","last_clean":"2024-09-05T13:13:01.168424+0000","last_became_active":"2024-09-05T13:12:24.165469+0000","last_became_peered":"2024-09-05T13:12:24.165469+0000","last_unstale":"2024-09-05T13:13:01.168424+0000","last_undegraded":"2024-09-05T13:13:01.168424+0000","last_fullsized":"2024-09-05T13:13:01.168424+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":99,"log_size":3487,"log_dups_size":0,"ondisk_log_size":3487,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14263238,"num_objects":3484,"num_object_clones":0,"num_object_copies":6968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3485,"num_write_kb":13931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'3459","reported_seq":3527,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:13:01.143949+0000","last_change":"2024-09-05T13:12:57.324469+0000","last_active":"2024-09-05T13:13:01.143949+0000","last_peered":"2024-09-05T13:13:01.143949+0000","last_clean":"2024-09-05T13:13:01.143949+0000","last_became_active":"2024-09-05T13:12:24.165604+0000","last_became_peered":"2024-09-05T13:12:24.165604+0000","last_unstale":"2024-09-05T13:13:01.143949+0000","last_undegraded":"2024-09-05T13:13:01.143949+0000","last_fullsized":"2024-09-05T13:13:01.143949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3459,"log_dups_size":0,"ondisk_log_size":3459,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14144454,"num_objects":3455,"num_object_clones":0,"num_object_copies":6910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3457,"num_write_kb":13816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'3512","reported_seq":3624,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:13:01.135834+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:01.135834+0000","last_peered":"2024-09-05T13:13:01.135834+0000","last_clean":"2024-09-05T13:13:01.135834+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:01.135834+0000","last_undegraded":"2024-09-05T13:13:01.135834+0000","last_fullsized":"2024-09-05T13:13:01.135834+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":3512,"log_dups_size":0,"ondisk_log_size":3512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14376960,"num_objects":3510,"num_object_clones":0,"num_object_copies":7020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122,"num_write_kb":488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.160861+0000","last_change":"2024-09-05T13:12:24.163255+0000","last_active":"2024-09-05T13:12:25.160861+0000","last_peered":"2024-09-05T13:12:25.160861+0000","last_clean":"2024-09-05T13:12:25.160861+0000","last_became_active":"2024-09-05T13:12:24.162958+0000","last_became_peered":"2024-09-05T13:12:24.162958+0000","last_unstale":"2024-09-05T13:12:25.160861+0000","last_undegraded":"2024-09-05T13:12:25.160861+0000","last_fullsized":"2024-09-05T13:12:25.160861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:58.405279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'3151","reported_seq":3256,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.138889+0000","last_change":"2024-09-05T13:12:49.050077+0000","last_active":"2024-09-05T13:12:57.138889+0000","last_peered":"2024-09-05T13:12:57.138889+0000","last_clean":"2024-09-05T13:12:57.138889+0000","last_became_active":"2024-09-05T13:12:25.167319+0000","last_became_peered":"2024-09-05T13:12:25.167319+0000","last_unstale":"2024-09-05T13:12:57.138889+0000","last_undegraded":"2024-09-05T13:12:57.138889+0000","last_fullsized":"2024-09-05T13:12:57.138889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":3151,"log_dups_size":0,"ondisk_log_size":3151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:33.891462+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12877824,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":805,"num_write_kb":3220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.165344+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:12:25.165344+0000","last_peered":"2024-09-05T13:12:25.165344+0000","last_clean":"2024-09-05T13:12:25.165344+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:12:25.165344+0000","last_undegraded":"2024-09-05T13:12:25.165344+0000","last_fullsized":"2024-09-05T13:12:25.165344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'3460","reported_seq":3521,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:13:01.115301+0000","last_change":"2024-09-05T13:12:57.258164+0000","last_active":"2024-09-05T13:13:01.115301+0000","last_peered":"2024-09-05T13:13:01.115301+0000","last_clean":"2024-09-05T13:13:01.115301+0000","last_became_active":"2024-09-05T13:12:24.165511+0000","last_became_peered":"2024-09-05T13:12:24.165511+0000","last_unstale":"2024-09-05T13:13:01.115301+0000","last_undegraded":"2024-09-05T13:13:01.115301+0000","last_fullsized":"2024-09-05T13:13:01.115301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14140358,"num_objects":3454,"num_object_clones":0,"num_object_copies":6908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3458,"num_write_kb":13812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":74,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.524104+0000","last_change":"2024-09-05T13:12:57.524013+0000","last_active":"2024-09-05T13:12:57.524104+0000","last_peered":"2024-09-05T13:12:57.524104+0000","last_clean":"2024-09-05T13:12:57.524104+0000","last_became_active":"2024-09-05T13:12:24.163048+0000","last_became_peered":"2024-09-05T13:12:24.163048+0000","last_unstale":"2024-09-05T13:12:57.524104+0000","last_undegraded":"2024-09-05T13:12:57.524104+0000","last_fullsized":"2024-09-05T13:12:57.524104+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:57:19.380472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'3099","reported_seq":3192,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.151915+0000","last_change":"2024-09-05T13:12:50.064282+0000","last_active":"2024-09-05T13:12:57.151915+0000","last_peered":"2024-09-05T13:12:57.151915+0000","last_clean":"2024-09-05T13:12:57.151915+0000","last_became_active":"2024-09-05T13:12:25.165724+0000","last_became_peered":"2024-09-05T13:12:25.165724+0000","last_unstale":"2024-09-05T13:12:57.151915+0000","last_undegraded":"2024-09-05T13:12:57.151915+0000","last_fullsized":"2024-09-05T13:12:57.151915+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":3099,"log_dups_size":0,"ondisk_log_size":3099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:47:05.084065+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12673024,"num_objects":3094,"num_object_clones":0,"num_object_copies":6188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":694,"num_write_kb":2776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.169048+0000","last_change":"2024-09-05T13:12:25.167242+0000","last_active":"2024-09-05T13:12:25.169048+0000","last_peered":"2024-09-05T13:12:25.169048+0000","last_clean":"2024-09-05T13:12:25.169048+0000","last_became_active":"2024-09-05T13:12:25.165964+0000","last_became_peered":"2024-09-05T13:12:25.165964+0000","last_unstale":"2024-09-05T13:12:25.169048+0000","last_undegraded":"2024-09-05T13:12:25.169048+0000","last_fullsized":"2024-09-05T13:12:25.169048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:03.155260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'3130","reported_seq":3226,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:57.099937+0000","last_change":"2024-09-05T13:12:51.179418+0000","last_active":"2024-09-05T13:12:57.099937+0000","last_peered":"2024-09-05T13:12:57.099937+0000","last_clean":"2024-09-05T13:12:57.099937+0000","last_became_active":"2024-09-05T13:12:25.164969+0000","last_became_peered":"2024-09-05T13:12:25.164969+0000","last_unstale":"2024-09-05T13:12:57.099937+0000","last_undegraded":"2024-09-05T13:12:57.099937+0000","last_fullsized":"2024-09-05T13:12:57.099937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":3130,"log_dups_size":0,"ondisk_log_size":3130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:28:28.361737+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12795904,"num_objects":3124,"num_object_clones":0,"num_object_copies":6248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":639,"num_write_kb":2556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'3471","reported_seq":3535,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:13:01.153569+0000","last_change":"2024-09-05T13:12:57.421789+0000","last_active":"2024-09-05T13:13:01.153569+0000","last_peered":"2024-09-05T13:13:01.153569+0000","last_clean":"2024-09-05T13:13:01.153569+0000","last_became_active":"2024-09-05T13:12:24.168300+0000","last_became_peered":"2024-09-05T13:12:24.168300+0000","last_unstale":"2024-09-05T13:13:01.153569+0000","last_undegraded":"2024-09-05T13:13:01.153569+0000","last_fullsized":"2024-09-05T13:13:01.153569+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3471,"log_dups_size":0,"ondisk_log_size":3471,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14209507,"num_objects":3470,"num_object_clones":0,"num_object_copies":6940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3470,"num_write_kb":13877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162822+0000","last_change":"2024-09-05T13:12:24.163476+0000","last_active":"2024-09-05T13:12:25.162822+0000","last_peered":"2024-09-05T13:12:25.162822+0000","last_clean":"2024-09-05T13:12:25.162822+0000","last_became_active":"2024-09-05T13:12:24.163169+0000","last_became_peered":"2024-09-05T13:12:24.163169+0000","last_unstale":"2024-09-05T13:12:25.162822+0000","last_undegraded":"2024-09-05T13:12:25.162822+0000","last_fullsized":"2024-09-05T13:12:25.162822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:05:50.037488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'3497","reported_seq":3566,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:13:01.167726+0000","last_change":"2024-09-05T13:12:57.500734+0000","last_active":"2024-09-05T13:13:01.167726+0000","last_peered":"2024-09-05T13:13:01.167726+0000","last_clean":"2024-09-05T13:13:01.167726+0000","last_became_active":"2024-09-05T13:12:24.168347+0000","last_became_peered":"2024-09-05T13:12:24.168347+0000","last_unstale":"2024-09-05T13:13:01.167726+0000","last_undegraded":"2024-09-05T13:13:01.167726+0000","last_fullsized":"2024-09-05T13:13:01.167726+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3497,"log_dups_size":0,"ondisk_log_size":3497,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14292393,"num_objects":3492,"num_object_clones":0,"num_object_copies":6984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3494,"num_write_kb":13960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.166730+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:12:25.166730+0000","last_peered":"2024-09-05T13:12:25.166730+0000","last_clean":"2024-09-05T13:12:25.166730+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:12:25.166730+0000","last_undegraded":"2024-09-05T13:12:25.166730+0000","last_fullsized":"2024-09-05T13:12:25.166730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'3582","reported_seq":3648,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:13:01.161347+0000","last_change":"2024-09-05T13:12:57.387742+0000","last_active":"2024-09-05T13:13:01.161347+0000","last_peered":"2024-09-05T13:13:01.161347+0000","last_clean":"2024-09-05T13:13:01.161347+0000","last_became_active":"2024-09-05T13:12:24.165544+0000","last_became_peered":"2024-09-05T13:12:24.165544+0000","last_unstale":"2024-09-05T13:13:01.161347+0000","last_undegraded":"2024-09-05T13:13:01.161347+0000","last_fullsized":"2024-09-05T13:13:01.161347+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14656454,"num_objects":3580,"num_object_clones":0,"num_object_copies":7160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3580,"num_write_kb":14314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.162476+0000","last_change":"2024-09-05T13:12:24.165771+0000","last_active":"2024-09-05T13:12:25.162476+0000","last_peered":"2024-09-05T13:12:25.162476+0000","last_clean":"2024-09-05T13:12:25.162476+0000","last_became_active":"2024-09-05T13:12:24.165476+0000","last_became_peered":"2024-09-05T13:12:24.165476+0000","last_unstale":"2024-09-05T13:12:25.162476+0000","last_undegraded":"2024-09-05T13:12:25.162476+0000","last_fullsized":"2024-09-05T13:12:25.162476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:09:24.468878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:12:25.168816+0000","last_change":"2024-09-05T13:12:25.166728+0000","last_active":"2024-09-05T13:12:25.168816+0000","last_peered":"2024-09-05T13:12:25.168816+0000","last_clean":"2024-09-05T13:12:25.168816+0000","last_became_active":"2024-09-05T13:12:25.166434+0000","last_became_peered":"2024-09-05T13:12:25.166434+0000","last_unstale":"2024-09-05T13:12:25.168816+0000","last_undegraded":"2024-09-05T13:12:25.168816+0000","last_fullsized":"2024-09-05T13:12:25.168816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:26:25.045325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'3570","reported_seq":3683,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-05T13:13:01.137393+0000","last_change":"2024-09-05T13:12:59.497780+0000","last_active":"2024-09-05T13:13:01.137393+0000","last_peered":"2024-09-05T13:13:01.137393+0000","last_clean":"2024-09-05T13:13:01.137393+0000","last_became_active":"2024-09-05T13:12:24.163127+0000","last_became_peered":"2024-09-05T13:12:24.163127+0000","last_unstale":"2024-09-05T13:13:01.137393+0000","last_undegraded":"2024-09-05T13:13:01.137393+0000","last_fullsized":"2024-09-05T13:13:01.137393+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:16:50.865261+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14574051,"num_objects":3559,"num_object_clones":0,"num_object_copies":7118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":219880525,"num_objects":53695,"num_object_clones":0,"num_object_copies":107390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35605,"num_write_kb":142338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434266112,"data_stored":434157722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53762,"ondisk_log_size":53762,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306724,"kb_used_data":194516,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94065116,"statfs":{"total":96636764160,"available":96322678784,"internally_reserved":0,"allocated":199184384,"data_stored":198976311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":114880173},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182417,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353348,"kb_used_data":223156,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94018492,"statfs":{"total":96636764160,"available":96274935808,"internally_reserved":0,"allocated":228511744,"data_stored":228295981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":133295786},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,5,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228491264,"data_stored":228487651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205660160,"data_stored":205656547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:03.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-05T13:13:03.243 INFO:tasks.util.rados:rados -p unfounddel_pool get new_1 - 2024-09-05T13:13:03.243 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_1 - 2024-09-05T13:13:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_1: (2) No such file or directory 2024-09-05T13:13:03.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:03.372 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_1 - 2024-09-05T13:13:03.372 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_1 - 2024-09-05T13:13:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_1: (2) No such file or directory 2024-09-05T13:13:03.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:03.485 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_1 - 2024-09-05T13:13:03.485 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_1 - 2024-09-05T13:13:03.580 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_1: (2) No such file or directory 2024-09-05T13:13:03.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:03.595 INFO:tasks.util.rados:rados -p unfounddel_pool get new_2 - 2024-09-05T13:13:03.595 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_2 - 2024-09-05T13:13:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_2: (2) No such file or directory 2024-09-05T13:13:03.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:03.726 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_2 - 2024-09-05T13:13:03.726 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_2 - 2024-09-05T13:13:03.825 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_2: (2) No such file or directory 2024-09-05T13:13:03.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:03.838 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_2 - 2024-09-05T13:13:03.838 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_2 - 2024-09-05T13:13:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_2: (2) No such file or directory 2024-09-05T13:13:03.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:03.950 INFO:tasks.util.rados:rados -p unfounddel_pool get new_3 - 2024-09-05T13:13:03.950 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_3 - 2024-09-05T13:13:04.067 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_3: (2) No such file or directory 2024-09-05T13:13:04.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:04.073 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_3 - 2024-09-05T13:13:04.073 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_3 - 2024-09-05T13:13:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_3: (2) No such file or directory 2024-09-05T13:13:04.189 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 31 20 59539 59519 7.49871 4.37109 0.0057764 0.0104114 2024-09-05T13:13:04.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:04.198 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_3 - 2024-09-05T13:13:04.198 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_3 - 2024-09-05T13:13:04.301 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_3: (2) No such file or directory 2024-09-05T13:13:04.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:04.313 INFO:tasks.util.rados:rados -p unfounddel_pool get new_4 - 2024-09-05T13:13:04.313 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_4 - 2024-09-05T13:13:04.433 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_4: (2) No such file or directory 2024-09-05T13:13:04.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:04.442 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_4 - 2024-09-05T13:13:04.442 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_4 - 2024-09-05T13:13:04.546 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_4: (2) No such file or directory 2024-09-05T13:13:04.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:04.564 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_4 - 2024-09-05T13:13:04.564 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_4 - 2024-09-05T13:13:04.670 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_4: (2) No such file or directory 2024-09-05T13:13:04.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:04.691 INFO:tasks.util.rados:rados -p unfounddel_pool get new_5 - 2024-09-05T13:13:04.691 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_5 - 2024-09-05T13:13:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_5: (2) No such file or directory 2024-09-05T13:13:04.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:04.809 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_5 - 2024-09-05T13:13:04.809 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_5 - 2024-09-05T13:13:04.921 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_5: (2) No such file or directory 2024-09-05T13:13:04.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:04.932 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_5 - 2024-09-05T13:13:04.932 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_5 - 2024-09-05T13:13:05.034 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_5: (2) No such file or directory 2024-09-05T13:13:05.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:05.040 INFO:tasks.util.rados:rados -p unfounddel_pool get new_6 - 2024-09-05T13:13:05.040 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_6 - 2024-09-05T13:13:05.147 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_6: (2) No such file or directory 2024-09-05T13:13:05.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:05.161 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_6 - 2024-09-05T13:13:05.161 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_6 - 2024-09-05T13:13:05.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 32 20 60698 60678 7.40585 4.52734 0.0148838 0.0105426 2024-09-05T13:13:05.273 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_6: (2) No such file or directory 2024-09-05T13:13:05.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:05.287 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_6 - 2024-09-05T13:13:05.287 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_6 - 2024-09-05T13:13:05.425 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_6: (2) No such file or directory 2024-09-05T13:13:05.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:05.438 INFO:tasks.util.rados:rados -p unfounddel_pool get new_7 - 2024-09-05T13:13:05.439 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_7 - 2024-09-05T13:13:05.545 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_7: (2) No such file or directory 2024-09-05T13:13:05.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:05.552 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_7 - 2024-09-05T13:13:05.552 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_7 - 2024-09-05T13:13:05.661 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_7: (2) No such file or directory 2024-09-05T13:13:05.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:05.676 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_7 - 2024-09-05T13:13:05.676 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_7 - 2024-09-05T13:13:05.786 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_7: (2) No such file or directory 2024-09-05T13:13:05.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:05.803 INFO:tasks.util.rados:rados -p unfounddel_pool get new_8 - 2024-09-05T13:13:05.803 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_8 - 2024-09-05T13:13:05.908 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_8: (2) No such file or directory 2024-09-05T13:13:05.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:05.925 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_8 - 2024-09-05T13:13:05.925 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_8 - 2024-09-05T13:13:06.026 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_8: (2) No such file or directory 2024-09-05T13:13:06.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:06.036 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_8 - 2024-09-05T13:13:06.036 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_8 - 2024-09-05T13:13:06.134 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_8: (2) No such file or directory 2024-09-05T13:13:06.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:06.141 INFO:tasks.util.rados:rados -p unfounddel_pool get new_9 - 2024-09-05T13:13:06.142 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_9 - 2024-09-05T13:13:06.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 33 20 61874 61854 7.32063 4.59375 0.0130794 0.0106656 2024-09-05T13:13:06.252 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/new_9: (2) No such file or directory 2024-09-05T13:13:06.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:06.263 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_9 - 2024-09-05T13:13:06.263 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_9 - 2024-09-05T13:13:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existed_9: (2) No such file or directory 2024-09-05T13:13:06.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:06.392 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_9 - 2024-09-05T13:13:06.392 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_9 - 2024-09-05T13:13:06.499 INFO:teuthology.orchestra.run.smithi086.stderr:error getting unfounddel_pool/existing_9: (2) No such file or directory 2024-09-05T13:13:06.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:13:06.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-05T13:13:07.087 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-09-05T13:13:07.085+0000 7f9a87bca640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-09-05T13:13:07.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 34 20 62976 62956 7.23192 4.30469 0.0141832 0.0107973 2024-09-05T13:13:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:marked in osd.1. 2024-09-05T13:13:07.766 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-05T13:13:07.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-05T13:13:07.768 INFO:tasks.ceph.osd.1:Started 2024-09-05T13:13:07.769 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-05T13:13:07.866 INFO:teuthology.orchestra.run.smithi086.stderr:no valid command found; 10 closest matches: 2024-09-05T13:13:07.866 INFO:teuthology.orchestra.run.smithi086.stderr:0 2024-09-05T13:13:07.866 INFO:teuthology.orchestra.run.smithi086.stderr:1 2024-09-05T13:13:07.866 INFO:teuthology.orchestra.run.smithi086.stderr:2 2024-09-05T13:13:07.866 INFO:teuthology.orchestra.run.smithi086.stderr:abort 2024-09-05T13:13:07.866 INFO:teuthology.orchestra.run.smithi086.stderr:assert 2024-09-05T13:13:07.866 INFO:teuthology.orchestra.run.smithi086.stderr:config diff 2024-09-05T13:13:07.867 INFO:teuthology.orchestra.run.smithi086.stderr:config diff get 2024-09-05T13:13:07.867 INFO:teuthology.orchestra.run.smithi086.stderr:config get 2024-09-05T13:13:07.867 INFO:teuthology.orchestra.run.smithi086.stderr:config help [] 2024-09-05T13:13:07.867 INFO:teuthology.orchestra.run.smithi086.stderr:config set ... 2024-09-05T13:13:07.867 INFO:teuthology.orchestra.run.smithi086.stderr:admin_socket: invalid command 2024-09-05T13:13:07.868 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-05T13:13:07.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-05T13:13:08.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 35 20 63709 63689 7.1071 2.86328 0.00639848 0.01085 2024-09-05T13:13:08.925 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-09-05T13:13:08.923+0000 7f9c31f4b640 -1 Falling back to public interface 2024-09-05T13:13:09.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 36 20 63709 63689 6.90969 0 - 0.01085 2024-09-05T13:13:10.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 37 20 64269 64249 6.78205 1.09375 0.0172911 0.0115131 2024-09-05T13:13:11.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 38 20 65529 65509 6.73309 4.92188 0.0252637 0.0115972 2024-09-05T13:13:12.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 39 20 66789 66769 6.68664 4.92188 0.0213304 0.0116784 2024-09-05T13:13:12.806 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-09-05T13:13:12.804+0000 7f9c31f4b640 -1 osd.1 26 log_to_monitors true 2024-09-05T13:13:12.870 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-05T13:13:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:no valid command found; 10 closest matches: 2024-09-05T13:13:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:0 2024-09-05T13:13:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:1 2024-09-05T13:13:13.037 INFO:teuthology.orchestra.run.smithi086.stderr:2 2024-09-05T13:13:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:abort 2024-09-05T13:13:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:assert 2024-09-05T13:13:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs debug_inject_read_zeros 2024-09-05T13:13:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs files list 2024-09-05T13:13:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:bluefs stats 2024-09-05T13:13:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:bluestore allocator dump block 2024-09-05T13:13:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:bluestore allocator fragmentation block 2024-09-05T13:13:13.038 INFO:teuthology.orchestra.run.smithi086.stderr:admin_socket: invalid command 2024-09-05T13:13:13.040 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-05T13:13:13.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-05T13:13:13.190 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:13:13.190678+0000 min lat: 0.00201413 max lat: 2.05855 avg lat: 0.0117531 2024-09-05T13:13:13.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:13:13.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 40 20 68064 68044 6.64398 4.98047 0.0215286 0.0117531 2024-09-05T13:13:14.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 41 20 69255 69235 6.5954 4.65234 0.0194404 0.0118386 2024-09-05T13:13:14.879 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-09-05T13:13:14.878+0000 7f9c25cf9640 -1 osd.1 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T13:13:15.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 42 20 70360 70340 6.54113 4.31641 0.00912064 0.0119378 2024-09-05T13:13:16.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 43 20 71275 71255 6.47213 3.57422 0.016911 0.0119852 2024-09-05T13:13:17.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 44 20 71275 71255 6.32504 0 - 0.0119852 2024-09-05T13:13:18.042 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-05T13:13:18.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 45 20 71275 71255 6.18449 0 - 0.0119852 2024-09-05T13:13:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:13:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "ops": [], 2024-09-05T13:13:18.234 INFO:teuthology.orchestra.run.smithi086.stdout: "num_ops": 0 2024-09-05T13:13:18.234 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:13:18.250 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.1 to be up 2024-09-05T13:13:18.251 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T13:13:18.740 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:18.741 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":43,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:13:17.895411+0000","last_up_change":"2024-09-05T13:13:15.880253+0000","last_in_change":"2024-09-05T13:13:07.720441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-05T13:11:28.452062+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-05T13:11:39.270087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":42,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6804","nonce":2531722421}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6805","nonce":2531722421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6807","nonce":2531722421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6806","nonce":2531722421}]},"public_addr":"172.21.15.86:6804/2531722421","cluster_addr":"172.21.15.86:6805/2531722421","heartbeat_back_addr":"172.21.15.86:6807/2531722421","heartbeat_front_addr":"172.21.15.86:6806/2531722421","state":["exists","up"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":26,"up_from":41,"up_thru":42,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6808","nonce":4255212274}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6809","nonce":4255212274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6811","nonce":4255212274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6810","nonce":4255212274}]},"public_addr":"172.21.15.86:6808/4255212274","cluster_addr":"172.21.15.86:6809/4255212274","heartbeat_back_addr":"172.21.15.86:6811/4255212274","heartbeat_front_addr":"172.21.15.86:6810/4255212274","state":["exists","up"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":17,"up_from":31,"up_thru":42,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2895960470}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2895960470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2895960470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2895960470}]},"public_addr":"172.21.15.86:6800/2895960470","cluster_addr":"172.21.15.86:6801/2895960470","heartbeat_back_addr":"172.21.15.86:6803/2895960470","heartbeat_front_addr":"172.21.15.86:6802/2895960470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.643488+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.440515+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:21.020419+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:13:18.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.1 is up 2024-09-05T13:13:18.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-05T13:13:18.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:19.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 46 20 71653 71633 6.08214 0.492188 0.0282701 0.0128364 2024-09-05T13:13:19.289 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:19.315 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-05T13:13:17.899695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":291910134,"num_objects":71270,"num_object_clones":0,"num_object_copies":142540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":71270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27069,"num_write_kb":108270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":12307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71352,"ondisk_log_size":71352,"up":46,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":874428,"kb_used_data":571140,"kb_used_omap":12,"kb_used_meta":303219,"kb_avail":282241092,"statfs":{"total":289910292480,"available":289014878208,"internally_reserved":0,"allocated":584847360,"data_stored":584107001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":310496262},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,9,7,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21420631,"num_objects":5227,"num_object_clones":0,"num_object_copies":10454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-49539,"num_objects_unfound":0,"num_objects_dirty":5227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.380506"},"pg_stats":[{"pgid":"2.f","version":"40'4421","reported_seq":4483,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.910891+0000","last_change":"2024-09-05T13:13:15.913790+0000","last_active":"2024-09-05T13:13:15.913790+0000","last_peered":"2024-09-05T13:13:15.910891+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:09.763743+0000","last_became_peered":"2024-09-05T13:13:09.763743+0000","last_unstale":"2024-09-05T13:13:15.910891+0000","last_undegraded":"2024-09-05T13:13:15.910891+0000","last_fullsized":"2024-09-05T13:13:15.910891+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18108416,"num_objects":4421,"num_object_clones":0,"num_object_copies":8842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4421,"num_write_kb":17684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4453","reported_seq":4550,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.909806+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:16.909806+0000","last_peered":"2024-09-05T13:13:16.909806+0000","last_clean":"2024-09-05T13:13:16.909806+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:16.909806+0000","last_undegraded":"2024-09-05T13:13:16.909806+0000","last_fullsized":"2024-09-05T13:13:16.909806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4496","reported_seq":4612,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.903012+0000","last_change":"2024-09-05T13:13:15.910044+0000","last_active":"2024-09-05T13:13:15.910044+0000","last_peered":"2024-09-05T13:13:15.874937+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:09.757821+0000","last_became_peered":"2024-09-05T13:13:09.757821+0000","last_unstale":"2024-09-05T13:13:16.903012+0000","last_undegraded":"2024-09-05T13:13:16.903012+0000","last_fullsized":"2024-09-05T13:13:16.903012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4496,"log_dups_size":0,"ondisk_log_size":4496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18399232,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2307,"num_write_kb":9228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4442","reported_seq":4504,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.904207+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:16.904207+0000","last_peered":"2024-09-05T13:13:16.904207+0000","last_clean":"2024-09-05T13:13:16.904207+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:16.904207+0000","last_undegraded":"2024-09-05T13:13:16.904207+0000","last_fullsized":"2024-09-05T13:13:16.904207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18194432,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'4438","reported_seq":4563,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-05T13:13:15.908634+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:15.908634+0000","last_peered":"2024-09-05T13:13:15.908634+0000","last_clean":"2024-09-05T13:13:15.908634+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:15.908634+0000","last_undegraded":"2024-09-05T13:13:15.908634+0000","last_fullsized":"2024-09-05T13:13:15.908634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4438,"log_dups_size":0,"ondisk_log_size":4438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'4456","reported_seq":4581,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.850773+0000","last_change":"2024-09-05T13:13:15.906028+0000","last_active":"2024-09-05T13:13:15.906028+0000","last_peered":"2024-09-05T13:13:15.850773+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:09.765681+0000","last_became_peered":"2024-09-05T13:13:09.765681+0000","last_unstale":"2024-09-05T13:13:15.850773+0000","last_undegraded":"2024-09-05T13:13:15.850773+0000","last_fullsized":"2024-09-05T13:13:15.850773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4456,"log_dups_size":0,"ondisk_log_size":4456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":911,"num_write_kb":3644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'4442","reported_seq":4568,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.884879+0000","last_change":"2024-09-05T13:13:15.913025+0000","last_active":"2024-09-05T13:13:15.913025+0000","last_peered":"2024-09-05T13:13:15.884879+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:09.756411+0000","last_became_peered":"2024-09-05T13:13:09.756411+0000","last_unstale":"2024-09-05T13:13:15.884879+0000","last_undegraded":"2024-09-05T13:13:15.884879+0000","last_fullsized":"2024-09-05T13:13:15.884879+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18169856,"num_objects":4436,"num_object_clones":0,"num_object_copies":8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4476","reported_seq":4598,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-05T13:13:15.912149+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:15.912149+0000","last_peered":"2024-09-05T13:13:15.912149+0000","last_clean":"2024-09-05T13:13:15.912149+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:15.912149+0000","last_undegraded":"2024-09-05T13:13:15.912149+0000","last_fullsized":"2024-09-05T13:13:15.912149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4476,"log_dups_size":0,"ondisk_log_size":4476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1086,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:13.775075+0000","last_change":"2024-09-05T13:13:15.905598+0000","last_active":"2024-09-05T13:13:15.905598+0000","last_peered":"2024-09-05T13:13:13.775075+0000","last_clean":"2024-09-05T13:13:13.775075+0000","last_became_active":"2024-09-05T13:13:09.756143+0000","last_became_peered":"2024-09-05T13:13:09.756143+0000","last_unstale":"2024-09-05T13:13:13.775075+0000","last_undegraded":"2024-09-05T13:13:13.775075+0000","last_fullsized":"2024-09-05T13:13:13.775075+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"40'4458","reported_seq":4581,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.903407+0000","last_change":"2024-09-05T13:13:15.908158+0000","last_active":"2024-09-05T13:13:15.908158+0000","last_peered":"2024-09-05T13:13:15.856125+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:09.774387+0000","last_became_peered":"2024-09-05T13:13:09.774387+0000","last_unstale":"2024-09-05T13:13:16.903407+0000","last_undegraded":"2024-09-05T13:13:16.903407+0000","last_fullsized":"2024-09-05T13:13:16.903407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.910327+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:16.910327+0000","last_peered":"2024-09-05T13:13:16.910327+0000","last_clean":"2024-09-05T13:13:16.910327+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:16.910327+0000","last_undegraded":"2024-09-05T13:13:16.910327+0000","last_fullsized":"2024-09-05T13:13:16.910327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'4462","reported_seq":4586,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.889399+0000","last_change":"2024-09-05T13:13:15.906584+0000","last_active":"2024-09-05T13:13:15.906584+0000","last_peered":"2024-09-05T13:13:15.889399+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:09.773402+0000","last_became_peered":"2024-09-05T13:13:09.773402+0000","last_unstale":"2024-09-05T13:13:15.889399+0000","last_undegraded":"2024-09-05T13:13:15.889399+0000","last_fullsized":"2024-09-05T13:13:15.889399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4462,"log_dups_size":0,"ondisk_log_size":4462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18243584,"num_objects":4454,"num_object_clones":0,"num_object_copies":8908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":3316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:13.777681+0000","last_change":"2024-09-05T13:13:15.907337+0000","last_active":"2024-09-05T13:13:15.907337+0000","last_peered":"2024-09-05T13:13:13.777681+0000","last_clean":"2024-09-05T13:13:13.777681+0000","last_became_active":"2024-09-05T13:13:09.770617+0000","last_became_peered":"2024-09-05T13:13:09.770617+0000","last_unstale":"2024-09-05T13:13:13.777681+0000","last_undegraded":"2024-09-05T13:13:13.777681+0000","last_fullsized":"2024-09-05T13:13:13.777681+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4392","reported_seq":4496,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.905660+0000","last_change":"2024-09-05T13:13:15.910392+0000","last_active":"2024-09-05T13:13:15.910392+0000","last_peered":"2024-09-05T13:13:15.895164+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:09.754496+0000","last_became_peered":"2024-09-05T13:13:09.754496+0000","last_unstale":"2024-09-05T13:13:16.905660+0000","last_undegraded":"2024-09-05T13:13:16.905660+0000","last_fullsized":"2024-09-05T13:13:16.905660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4392,"log_dups_size":0,"ondisk_log_size":4392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1987,"num_write_kb":7948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":30,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-05T13:13:09.773651+0000","last_change":"2024-09-05T13:13:09.773526+0000","last_active":"2024-09-05T13:13:09.773651+0000","last_peered":"2024-09-05T13:13:09.773651+0000","last_clean":"2024-09-05T13:13:09.773651+0000","last_became_active":"2024-09-05T13:13:09.773154+0000","last_became_peered":"2024-09-05T13:13:09.773154+0000","last_unstale":"2024-09-05T13:13:09.773651+0000","last_undegraded":"2024-09-05T13:13:09.773651+0000","last_fullsized":"2024-09-05T13:13:09.773651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:23:28.234959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"40'4393","reported_seq":4504,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.905522+0000","last_change":"2024-09-05T13:13:15.909438+0000","last_active":"2024-09-05T13:13:15.909438+0000","last_peered":"2024-09-05T13:13:15.890176+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:09.754566+0000","last_became_peered":"2024-09-05T13:13:09.754566+0000","last_unstale":"2024-09-05T13:13:16.905522+0000","last_undegraded":"2024-09-05T13:13:16.905522+0000","last_fullsized":"2024-09-05T13:13:16.905522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'4421","reported_seq":4548,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.895786+0000","last_change":"2024-09-05T13:13:15.916262+0000","last_active":"2024-09-05T13:13:15.916262+0000","last_peered":"2024-09-05T13:13:15.895786+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:09.764049+0000","last_became_peered":"2024-09-05T13:13:09.764049+0000","last_unstale":"2024-09-05T13:13:15.895786+0000","last_undegraded":"2024-09-05T13:13:15.895786+0000","last_fullsized":"2024-09-05T13:13:15.895786+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18100224,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":572,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:13.775712+0000","last_change":"2024-09-05T13:13:15.906510+0000","last_active":"2024-09-05T13:13:15.906510+0000","last_peered":"2024-09-05T13:13:13.775712+0000","last_clean":"2024-09-05T13:13:13.775712+0000","last_became_active":"2024-09-05T13:13:09.756334+0000","last_became_peered":"2024-09-05T13:13:09.756334+0000","last_unstale":"2024-09-05T13:13:13.775712+0000","last_undegraded":"2024-09-05T13:13:13.775712+0000","last_fullsized":"2024-09-05T13:13:13.775712+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"40'4452","reported_seq":4626,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.904984+0000","last_change":"2024-09-05T13:13:15.906706+0000","last_active":"2024-09-05T13:13:15.906706+0000","last_peered":"2024-09-05T13:13:15.904984+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:09.764187+0000","last_became_peered":"2024-09-05T13:13:09.764187+0000","last_unstale":"2024-09-05T13:13:15.904984+0000","last_undegraded":"2024-09-05T13:13:15.904984+0000","last_fullsized":"2024-09-05T13:13:15.904984+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4452,"log_dups_size":0,"ondisk_log_size":4452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18202624,"num_objects":4444,"num_object_clones":0,"num_object_copies":8888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":351,"num_write_kb":1404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.910577+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:16.910577+0000","last_peered":"2024-09-05T13:13:16.910577+0000","last_clean":"2024-09-05T13:13:16.910577+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:16.910577+0000","last_undegraded":"2024-09-05T13:13:16.910577+0000","last_fullsized":"2024-09-05T13:13:16.910577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4570","reported_seq":4695,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.888158+0000","last_change":"2024-09-05T13:13:15.906178+0000","last_active":"2024-09-05T13:13:15.906178+0000","last_peered":"2024-09-05T13:13:15.888158+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:09.758010+0000","last_became_peered":"2024-09-05T13:13:09.758010+0000","last_unstale":"2024-09-05T13:13:15.888158+0000","last_undegraded":"2024-09-05T13:13:15.888158+0000","last_fullsized":"2024-09-05T13:13:15.888158+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4570,"log_dups_size":0,"ondisk_log_size":4570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702336,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":700,"num_write_kb":2800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:13.776415+0000","last_change":"2024-09-05T13:13:15.905720+0000","last_active":"2024-09-05T13:13:15.905720+0000","last_peered":"2024-09-05T13:13:13.776415+0000","last_clean":"2024-09-05T13:13:13.776415+0000","last_became_active":"2024-09-05T13:13:09.766332+0000","last_became_peered":"2024-09-05T13:13:09.766332+0000","last_unstale":"2024-09-05T13:13:13.776415+0000","last_undegraded":"2024-09-05T13:13:13.776415+0000","last_fullsized":"2024-09-05T13:13:13.776415+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-05T13:13:09.762363+0000","last_change":"2024-09-05T13:13:09.762230+0000","last_active":"2024-09-05T13:13:09.762363+0000","last_peered":"2024-09-05T13:13:09.762363+0000","last_clean":"2024-09-05T13:13:09.762363+0000","last_became_active":"2024-09-05T13:13:09.761844+0000","last_became_peered":"2024-09-05T13:13:09.761844+0000","last_unstale":"2024-09-05T13:13:09.762363+0000","last_undegraded":"2024-09-05T13:13:09.762363+0000","last_fullsized":"2024-09-05T13:13:09.762363+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:34:26.559925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"40'4577","reported_seq":4703,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.904752+0000","last_change":"2024-09-05T13:13:15.908907+0000","last_active":"2024-09-05T13:13:15.908907+0000","last_peered":"2024-09-05T13:13:15.904752+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:09.764596+0000","last_became_peered":"2024-09-05T13:13:09.764596+0000","last_unstale":"2024-09-05T13:13:15.904752+0000","last_undegraded":"2024-09-05T13:13:15.904752+0000","last_fullsized":"2024-09-05T13:13:15.904752+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698723,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1245,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":291910115,"num_objects":71268,"num_object_clones":0,"num_object_copies":142536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27067,"num_write_kb":108268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":12288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":583925760,"data_stored":583817370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71349,"ondisk_log_size":71349,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425392,"kb_used_data":285404,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93946448,"statfs":{"total":96636764160,"available":96201162752,"internally_reserved":0,"allocated":292253696,"data_stored":292014445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421168,"kb_used_data":285404,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93950672,"statfs":{"total":96636764160,"available":96205488128,"internally_reserved":0,"allocated":292253696,"data_stored":292014445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,4,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":339968,"data_stored":78111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":28176047},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:19.318 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:19.888 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:19.889 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:19.912 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-05T13:13:19.523544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":291910134,"num_objects":71270,"num_object_clones":0,"num_object_copies":142540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27069,"num_write_kb":108270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":12307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71352,"ondisk_log_size":71352,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":911176,"kb_used_data":571780,"kb_used_omap":12,"kb_used_meta":339315,"kb_avail":282204344,"statfs":{"total":289910292480,"available":288977248256,"internally_reserved":0,"allocated":585502720,"data_stored":584695565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":347458566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,14,10,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21420631,"num_objects":5227,"num_object_clones":0,"num_object_copies":10454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-49540,"num_objects_unfound":0,"num_objects_dirty":5227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.760225"},"pg_stats":[{"pgid":"2.f","version":"40'4421","reported_seq":4483,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.910891+0000","last_change":"2024-09-05T13:13:15.913790+0000","last_active":"2024-09-05T13:13:15.913790+0000","last_peered":"2024-09-05T13:13:15.910891+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:09.763743+0000","last_became_peered":"2024-09-05T13:13:09.763743+0000","last_unstale":"2024-09-05T13:13:15.910891+0000","last_undegraded":"2024-09-05T13:13:15.910891+0000","last_fullsized":"2024-09-05T13:13:15.910891+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18108416,"num_objects":4421,"num_object_clones":0,"num_object_copies":8842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4421,"num_write_kb":17684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4453","reported_seq":4550,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.909806+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:16.909806+0000","last_peered":"2024-09-05T13:13:16.909806+0000","last_clean":"2024-09-05T13:13:16.909806+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:16.909806+0000","last_undegraded":"2024-09-05T13:13:16.909806+0000","last_fullsized":"2024-09-05T13:13:16.909806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4496","reported_seq":4612,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.903012+0000","last_change":"2024-09-05T13:13:15.910044+0000","last_active":"2024-09-05T13:13:15.910044+0000","last_peered":"2024-09-05T13:13:15.874937+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:09.757821+0000","last_became_peered":"2024-09-05T13:13:09.757821+0000","last_unstale":"2024-09-05T13:13:16.903012+0000","last_undegraded":"2024-09-05T13:13:16.903012+0000","last_fullsized":"2024-09-05T13:13:16.903012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4496,"log_dups_size":0,"ondisk_log_size":4496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18399232,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2307,"num_write_kb":9228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4442","reported_seq":4504,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.904207+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:16.904207+0000","last_peered":"2024-09-05T13:13:16.904207+0000","last_clean":"2024-09-05T13:13:16.904207+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:16.904207+0000","last_undegraded":"2024-09-05T13:13:16.904207+0000","last_fullsized":"2024-09-05T13:13:16.904207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18194432,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'4438","reported_seq":4563,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-05T13:13:15.908634+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:15.908634+0000","last_peered":"2024-09-05T13:13:15.908634+0000","last_clean":"2024-09-05T13:13:15.908634+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:15.908634+0000","last_undegraded":"2024-09-05T13:13:15.908634+0000","last_fullsized":"2024-09-05T13:13:15.908634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4438,"log_dups_size":0,"ondisk_log_size":4438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'4456","reported_seq":4581,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.850773+0000","last_change":"2024-09-05T13:13:15.906028+0000","last_active":"2024-09-05T13:13:15.906028+0000","last_peered":"2024-09-05T13:13:15.850773+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:09.765681+0000","last_became_peered":"2024-09-05T13:13:09.765681+0000","last_unstale":"2024-09-05T13:13:15.850773+0000","last_undegraded":"2024-09-05T13:13:15.850773+0000","last_fullsized":"2024-09-05T13:13:15.850773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4456,"log_dups_size":0,"ondisk_log_size":4456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":911,"num_write_kb":3644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'4442","reported_seq":4568,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.884879+0000","last_change":"2024-09-05T13:13:15.913025+0000","last_active":"2024-09-05T13:13:15.913025+0000","last_peered":"2024-09-05T13:13:15.884879+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:09.756411+0000","last_became_peered":"2024-09-05T13:13:09.756411+0000","last_unstale":"2024-09-05T13:13:15.884879+0000","last_undegraded":"2024-09-05T13:13:15.884879+0000","last_fullsized":"2024-09-05T13:13:15.884879+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18169856,"num_objects":4436,"num_object_clones":0,"num_object_copies":8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4476","reported_seq":4598,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-05T13:13:15.912149+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:15.912149+0000","last_peered":"2024-09-05T13:13:15.912149+0000","last_clean":"2024-09-05T13:13:15.912149+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:15.912149+0000","last_undegraded":"2024-09-05T13:13:15.912149+0000","last_fullsized":"2024-09-05T13:13:15.912149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4476,"log_dups_size":0,"ondisk_log_size":4476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1086,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"40'4458","reported_seq":4581,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.903407+0000","last_change":"2024-09-05T13:13:15.908158+0000","last_active":"2024-09-05T13:13:15.908158+0000","last_peered":"2024-09-05T13:13:15.856125+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:09.774387+0000","last_became_peered":"2024-09-05T13:13:09.774387+0000","last_unstale":"2024-09-05T13:13:16.903407+0000","last_undegraded":"2024-09-05T13:13:16.903407+0000","last_fullsized":"2024-09-05T13:13:16.903407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.910327+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:16.910327+0000","last_peered":"2024-09-05T13:13:16.910327+0000","last_clean":"2024-09-05T13:13:16.910327+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:16.910327+0000","last_undegraded":"2024-09-05T13:13:16.910327+0000","last_fullsized":"2024-09-05T13:13:16.910327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'4462","reported_seq":4586,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.889399+0000","last_change":"2024-09-05T13:13:15.906584+0000","last_active":"2024-09-05T13:13:15.906584+0000","last_peered":"2024-09-05T13:13:15.889399+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:09.773402+0000","last_became_peered":"2024-09-05T13:13:09.773402+0000","last_unstale":"2024-09-05T13:13:15.889399+0000","last_undegraded":"2024-09-05T13:13:15.889399+0000","last_fullsized":"2024-09-05T13:13:15.889399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4462,"log_dups_size":0,"ondisk_log_size":4462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18243584,"num_objects":4454,"num_object_clones":0,"num_object_copies":8908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":3316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:13.777681+0000","last_change":"2024-09-05T13:13:15.907337+0000","last_active":"2024-09-05T13:13:15.907337+0000","last_peered":"2024-09-05T13:13:13.777681+0000","last_clean":"2024-09-05T13:13:13.777681+0000","last_became_active":"2024-09-05T13:13:09.770617+0000","last_became_peered":"2024-09-05T13:13:09.770617+0000","last_unstale":"2024-09-05T13:13:13.777681+0000","last_undegraded":"2024-09-05T13:13:13.777681+0000","last_fullsized":"2024-09-05T13:13:13.777681+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4392","reported_seq":4496,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.905660+0000","last_change":"2024-09-05T13:13:15.910392+0000","last_active":"2024-09-05T13:13:15.910392+0000","last_peered":"2024-09-05T13:13:15.895164+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:09.754496+0000","last_became_peered":"2024-09-05T13:13:09.754496+0000","last_unstale":"2024-09-05T13:13:16.905660+0000","last_undegraded":"2024-09-05T13:13:16.905660+0000","last_fullsized":"2024-09-05T13:13:16.905660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4392,"log_dups_size":0,"ondisk_log_size":4392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1987,"num_write_kb":7948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"40'4393","reported_seq":4504,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.905522+0000","last_change":"2024-09-05T13:13:15.909438+0000","last_active":"2024-09-05T13:13:15.909438+0000","last_peered":"2024-09-05T13:13:15.890176+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:09.754566+0000","last_became_peered":"2024-09-05T13:13:09.754566+0000","last_unstale":"2024-09-05T13:13:16.905522+0000","last_undegraded":"2024-09-05T13:13:16.905522+0000","last_fullsized":"2024-09-05T13:13:16.905522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'4421","reported_seq":4548,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.895786+0000","last_change":"2024-09-05T13:13:15.916262+0000","last_active":"2024-09-05T13:13:15.916262+0000","last_peered":"2024-09-05T13:13:15.895786+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:09.764049+0000","last_became_peered":"2024-09-05T13:13:09.764049+0000","last_unstale":"2024-09-05T13:13:15.895786+0000","last_undegraded":"2024-09-05T13:13:15.895786+0000","last_fullsized":"2024-09-05T13:13:15.895786+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18100224,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":572,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"40'4452","reported_seq":4626,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.904984+0000","last_change":"2024-09-05T13:13:15.906706+0000","last_active":"2024-09-05T13:13:15.906706+0000","last_peered":"2024-09-05T13:13:15.904984+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:09.764187+0000","last_became_peered":"2024-09-05T13:13:09.764187+0000","last_unstale":"2024-09-05T13:13:15.904984+0000","last_undegraded":"2024-09-05T13:13:15.904984+0000","last_fullsized":"2024-09-05T13:13:15.904984+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4452,"log_dups_size":0,"ondisk_log_size":4452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18202624,"num_objects":4444,"num_object_clones":0,"num_object_copies":8888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":351,"num_write_kb":1404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.910577+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:16.910577+0000","last_peered":"2024-09-05T13:13:16.910577+0000","last_clean":"2024-09-05T13:13:16.910577+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:16.910577+0000","last_undegraded":"2024-09-05T13:13:16.910577+0000","last_fullsized":"2024-09-05T13:13:16.910577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4570","reported_seq":4695,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.888158+0000","last_change":"2024-09-05T13:13:15.906178+0000","last_active":"2024-09-05T13:13:15.906178+0000","last_peered":"2024-09-05T13:13:15.888158+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:09.758010+0000","last_became_peered":"2024-09-05T13:13:09.758010+0000","last_unstale":"2024-09-05T13:13:15.888158+0000","last_undegraded":"2024-09-05T13:13:15.888158+0000","last_fullsized":"2024-09-05T13:13:15.888158+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4570,"log_dups_size":0,"ondisk_log_size":4570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702336,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":700,"num_write_kb":2800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"40'4577","reported_seq":4703,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.904752+0000","last_change":"2024-09-05T13:13:15.908907+0000","last_active":"2024-09-05T13:13:15.908907+0000","last_peered":"2024-09-05T13:13:15.904752+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:09.764596+0000","last_became_peered":"2024-09-05T13:13:09.764596+0000","last_unstale":"2024-09-05T13:13:15.904752+0000","last_undegraded":"2024-09-05T13:13:15.904752+0000","last_fullsized":"2024-09-05T13:13:15.904752+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698723,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1245,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":291910115,"num_objects":71268,"num_object_clones":0,"num_object_copies":142536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27067,"num_write_kb":108268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":12288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":584478720,"data_stored":584388395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71349,"ondisk_log_size":71349,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425392,"kb_used_data":285404,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93946448,"statfs":{"total":96636764160,"available":96201162752,"internally_reserved":0,"allocated":292253696,"data_stored":292014445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421168,"kb_used_data":285404,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93950672,"statfs":{"total":96636764160,"available":96205488128,"internally_reserved":0,"allocated":292253696,"data_stored":292014445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,4,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64616,"kb_used_data":972,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94307224,"statfs":{"total":96636764160,"available":96570597376,"internally_reserved":0,"allocated":995328,"data_stored":666675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":584549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:19.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:20.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 47 20 72226 72206 6.00036 2.23828 0.0155281 0.0130107 2024-09-05T13:13:20.506 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:20.536 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-05T13:13:19.523544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":291910134,"num_objects":71270,"num_object_clones":0,"num_object_copies":142540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27069,"num_write_kb":108270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":12307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71352,"ondisk_log_size":71352,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":911176,"kb_used_data":571780,"kb_used_omap":12,"kb_used_meta":339315,"kb_avail":282204344,"statfs":{"total":289910292480,"available":288977248256,"internally_reserved":0,"allocated":585502720,"data_stored":584695565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":347458566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,14,10,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21420631,"num_objects":5227,"num_object_clones":0,"num_object_copies":10454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-49540,"num_objects_unfound":0,"num_objects_dirty":5227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.760225"},"pg_stats":[{"pgid":"2.f","version":"40'4421","reported_seq":4483,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.910891+0000","last_change":"2024-09-05T13:13:15.913790+0000","last_active":"2024-09-05T13:13:15.913790+0000","last_peered":"2024-09-05T13:13:15.910891+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:09.763743+0000","last_became_peered":"2024-09-05T13:13:09.763743+0000","last_unstale":"2024-09-05T13:13:15.910891+0000","last_undegraded":"2024-09-05T13:13:15.910891+0000","last_fullsized":"2024-09-05T13:13:15.910891+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18108416,"num_objects":4421,"num_object_clones":0,"num_object_copies":8842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4421,"num_write_kb":17684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4453","reported_seq":4550,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.909806+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:16.909806+0000","last_peered":"2024-09-05T13:13:16.909806+0000","last_clean":"2024-09-05T13:13:16.909806+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:16.909806+0000","last_undegraded":"2024-09-05T13:13:16.909806+0000","last_fullsized":"2024-09-05T13:13:16.909806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4496","reported_seq":4612,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.903012+0000","last_change":"2024-09-05T13:13:15.910044+0000","last_active":"2024-09-05T13:13:15.910044+0000","last_peered":"2024-09-05T13:13:15.874937+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:09.757821+0000","last_became_peered":"2024-09-05T13:13:09.757821+0000","last_unstale":"2024-09-05T13:13:16.903012+0000","last_undegraded":"2024-09-05T13:13:16.903012+0000","last_fullsized":"2024-09-05T13:13:16.903012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4496,"log_dups_size":0,"ondisk_log_size":4496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18399232,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2307,"num_write_kb":9228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4442","reported_seq":4504,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.904207+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:16.904207+0000","last_peered":"2024-09-05T13:13:16.904207+0000","last_clean":"2024-09-05T13:13:16.904207+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:16.904207+0000","last_undegraded":"2024-09-05T13:13:16.904207+0000","last_fullsized":"2024-09-05T13:13:16.904207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18194432,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'4438","reported_seq":4563,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-05T13:13:15.908634+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:15.908634+0000","last_peered":"2024-09-05T13:13:15.908634+0000","last_clean":"2024-09-05T13:13:15.908634+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:15.908634+0000","last_undegraded":"2024-09-05T13:13:15.908634+0000","last_fullsized":"2024-09-05T13:13:15.908634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4438,"log_dups_size":0,"ondisk_log_size":4438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18149376,"num_objects":4431,"num_object_clones":0,"num_object_copies":8862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'4456","reported_seq":4581,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.850773+0000","last_change":"2024-09-05T13:13:15.906028+0000","last_active":"2024-09-05T13:13:15.906028+0000","last_peered":"2024-09-05T13:13:15.850773+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:09.765681+0000","last_became_peered":"2024-09-05T13:13:09.765681+0000","last_unstale":"2024-09-05T13:13:15.850773+0000","last_undegraded":"2024-09-05T13:13:15.850773+0000","last_fullsized":"2024-09-05T13:13:15.850773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4456,"log_dups_size":0,"ondisk_log_size":4456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":911,"num_write_kb":3644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'4442","reported_seq":4568,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.884879+0000","last_change":"2024-09-05T13:13:15.913025+0000","last_active":"2024-09-05T13:13:15.913025+0000","last_peered":"2024-09-05T13:13:15.884879+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:09.756411+0000","last_became_peered":"2024-09-05T13:13:09.756411+0000","last_unstale":"2024-09-05T13:13:15.884879+0000","last_undegraded":"2024-09-05T13:13:15.884879+0000","last_fullsized":"2024-09-05T13:13:15.884879+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18169856,"num_objects":4436,"num_object_clones":0,"num_object_copies":8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4476","reported_seq":4598,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-05T13:13:15.912149+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:15.912149+0000","last_peered":"2024-09-05T13:13:15.912149+0000","last_clean":"2024-09-05T13:13:15.912149+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:15.912149+0000","last_undegraded":"2024-09-05T13:13:15.912149+0000","last_fullsized":"2024-09-05T13:13:15.912149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4476,"log_dups_size":0,"ondisk_log_size":4476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1086,"num_write_kb":4344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"40'4458","reported_seq":4581,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.903407+0000","last_change":"2024-09-05T13:13:15.908158+0000","last_active":"2024-09-05T13:13:15.908158+0000","last_peered":"2024-09-05T13:13:15.856125+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:09.774387+0000","last_became_peered":"2024-09-05T13:13:09.774387+0000","last_unstale":"2024-09-05T13:13:16.903407+0000","last_undegraded":"2024-09-05T13:13:16.903407+0000","last_fullsized":"2024-09-05T13:13:16.903407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.910327+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:16.910327+0000","last_peered":"2024-09-05T13:13:16.910327+0000","last_clean":"2024-09-05T13:13:16.910327+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:16.910327+0000","last_undegraded":"2024-09-05T13:13:16.910327+0000","last_fullsized":"2024-09-05T13:13:16.910327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'4462","reported_seq":4586,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.889399+0000","last_change":"2024-09-05T13:13:15.906584+0000","last_active":"2024-09-05T13:13:15.906584+0000","last_peered":"2024-09-05T13:13:15.889399+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:09.773402+0000","last_became_peered":"2024-09-05T13:13:09.773402+0000","last_unstale":"2024-09-05T13:13:15.889399+0000","last_undegraded":"2024-09-05T13:13:15.889399+0000","last_fullsized":"2024-09-05T13:13:15.889399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4462,"log_dups_size":0,"ondisk_log_size":4462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18243584,"num_objects":4454,"num_object_clones":0,"num_object_copies":8908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":829,"num_write_kb":3316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:13.777681+0000","last_change":"2024-09-05T13:13:15.907337+0000","last_active":"2024-09-05T13:13:15.907337+0000","last_peered":"2024-09-05T13:13:13.777681+0000","last_clean":"2024-09-05T13:13:13.777681+0000","last_became_active":"2024-09-05T13:13:09.770617+0000","last_became_peered":"2024-09-05T13:13:09.770617+0000","last_unstale":"2024-09-05T13:13:13.777681+0000","last_undegraded":"2024-09-05T13:13:13.777681+0000","last_fullsized":"2024-09-05T13:13:13.777681+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4392","reported_seq":4496,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.905660+0000","last_change":"2024-09-05T13:13:15.910392+0000","last_active":"2024-09-05T13:13:15.910392+0000","last_peered":"2024-09-05T13:13:15.895164+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:09.754496+0000","last_became_peered":"2024-09-05T13:13:09.754496+0000","last_unstale":"2024-09-05T13:13:16.905660+0000","last_undegraded":"2024-09-05T13:13:16.905660+0000","last_fullsized":"2024-09-05T13:13:16.905660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4392,"log_dups_size":0,"ondisk_log_size":4392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1987,"num_write_kb":7948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"40'4393","reported_seq":4504,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.905522+0000","last_change":"2024-09-05T13:13:15.909438+0000","last_active":"2024-09-05T13:13:15.909438+0000","last_peered":"2024-09-05T13:13:15.890176+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:09.754566+0000","last_became_peered":"2024-09-05T13:13:09.754566+0000","last_unstale":"2024-09-05T13:13:16.905522+0000","last_undegraded":"2024-09-05T13:13:16.905522+0000","last_fullsized":"2024-09-05T13:13:16.905522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'4421","reported_seq":4548,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.895786+0000","last_change":"2024-09-05T13:13:15.916262+0000","last_active":"2024-09-05T13:13:15.916262+0000","last_peered":"2024-09-05T13:13:15.895786+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:09.764049+0000","last_became_peered":"2024-09-05T13:13:09.764049+0000","last_unstale":"2024-09-05T13:13:15.895786+0000","last_undegraded":"2024-09-05T13:13:15.895786+0000","last_fullsized":"2024-09-05T13:13:15.895786+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18100224,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":572,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"40'4452","reported_seq":4626,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.904984+0000","last_change":"2024-09-05T13:13:15.906706+0000","last_active":"2024-09-05T13:13:15.906706+0000","last_peered":"2024-09-05T13:13:15.904984+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:09.764187+0000","last_became_peered":"2024-09-05T13:13:09.764187+0000","last_unstale":"2024-09-05T13:13:15.904984+0000","last_undegraded":"2024-09-05T13:13:15.904984+0000","last_fullsized":"2024-09-05T13:13:15.904984+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4452,"log_dups_size":0,"ondisk_log_size":4452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18202624,"num_objects":4444,"num_object_clones":0,"num_object_copies":8888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":351,"num_write_kb":1404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.910577+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:16.910577+0000","last_peered":"2024-09-05T13:13:16.910577+0000","last_clean":"2024-09-05T13:13:16.910577+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:16.910577+0000","last_undegraded":"2024-09-05T13:13:16.910577+0000","last_fullsized":"2024-09-05T13:13:16.910577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4570","reported_seq":4695,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.888158+0000","last_change":"2024-09-05T13:13:15.906178+0000","last_active":"2024-09-05T13:13:15.906178+0000","last_peered":"2024-09-05T13:13:15.888158+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:09.758010+0000","last_became_peered":"2024-09-05T13:13:09.758010+0000","last_unstale":"2024-09-05T13:13:15.888158+0000","last_undegraded":"2024-09-05T13:13:15.888158+0000","last_fullsized":"2024-09-05T13:13:15.888158+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4570,"log_dups_size":0,"ondisk_log_size":4570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702336,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":700,"num_write_kb":2800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"40'4577","reported_seq":4703,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:15.904752+0000","last_change":"2024-09-05T13:13:15.908907+0000","last_active":"2024-09-05T13:13:15.908907+0000","last_peered":"2024-09-05T13:13:15.904752+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:09.764596+0000","last_became_peered":"2024-09-05T13:13:09.764596+0000","last_unstale":"2024-09-05T13:13:15.904752+0000","last_undegraded":"2024-09-05T13:13:15.904752+0000","last_fullsized":"2024-09-05T13:13:15.904752+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698723,"num_objects":4566,"num_object_clones":0,"num_object_copies":9132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1245,"num_write_kb":4980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":291910115,"num_objects":71268,"num_object_clones":0,"num_object_copies":142536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27067,"num_write_kb":108268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":12288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":584478720,"data_stored":584388395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71349,"ondisk_log_size":71349,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425392,"kb_used_data":285404,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93946448,"statfs":{"total":96636764160,"available":96201162752,"internally_reserved":0,"allocated":292253696,"data_stored":292014445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421168,"kb_used_data":285404,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93950672,"statfs":{"total":96636764160,"available":96205488128,"internally_reserved":0,"allocated":292253696,"data_stored":292014445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,4,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64616,"kb_used_data":972,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94307224,"statfs":{"total":96636764160,"available":96570597376,"internally_reserved":0,"allocated":995328,"data_stored":666675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":584549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:20.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:13:20.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "40'4421", 'reported_seq': 4483, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:15.910891+0000', 'last_change': '2024-09-05T13:13:15.913790+0000', 'last_active': '2024-09-05T13:13:15.913790+0000', 'last_peered': '2024-09-05T13:13:15.910891+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:09.763743+0000', 'last_became_peered': '2024-09-05T13:13:09.763743+0000', 'last_unstale': '2024-09-05T13:13:15.910891+0000', 'last_undegraded': '2024-09-05T13:13:15.910891+0000', 'last_fullsized': '2024-09-05T13:13:15.910891+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 4421, 'log_dups_size': 0, 'ondisk_log_size': 4421, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18108416, 'num_objects': 4421, 'num_object_clones': 0, 'num_object_copies': 8842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4421, 'num_write_kb': 17684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:13:20.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "40'4496", 'reported_seq': 4612, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:16.903012+0000', 'last_change': '2024-09-05T13:13:15.910044+0000', 'last_active': '2024-09-05T13:13:15.910044+0000', 'last_peered': '2024-09-05T13:13:15.874937+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:09.757821+0000', 'last_became_peered': '2024-09-05T13:13:09.757821+0000', 'last_unstale': '2024-09-05T13:13:16.903012+0000', 'last_undegraded': '2024-09-05T13:13:16.903012+0000', 'last_fullsized': '2024-09-05T13:13:16.903012+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 4496, 'log_dups_size': 0, 'ondisk_log_size': 4496, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18399232, 'num_objects': 4492, 'num_object_clones': 0, 'num_object_copies': 8984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2307, 'num_write_kb': 9228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:20.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:13:20.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "40'4456", 'reported_seq': 4581, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:15.850773+0000', 'last_change': '2024-09-05T13:13:15.906028+0000', 'last_active': '2024-09-05T13:13:15.906028+0000', 'last_peered': '2024-09-05T13:13:15.850773+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:09.765681+0000', 'last_became_peered': '2024-09-05T13:13:09.765681+0000', 'last_unstale': '2024-09-05T13:13:15.850773+0000', 'last_undegraded': '2024-09-05T13:13:15.850773+0000', 'last_fullsized': '2024-09-05T13:13:15.850773+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 4456, 'log_dups_size': 0, 'ondisk_log_size': 4456, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18231296, 'num_objects': 4451, 'num_object_clones': 0, 'num_object_copies': 8902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 911, 'num_write_kb': 3644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:13:20.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "40'4442", 'reported_seq': 4568, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:15.884879+0000', 'last_change': '2024-09-05T13:13:15.913025+0000', 'last_active': '2024-09-05T13:13:15.913025+0000', 'last_peered': '2024-09-05T13:13:15.884879+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:09.756411+0000', 'last_became_peered': '2024-09-05T13:13:09.756411+0000', 'last_unstale': '2024-09-05T13:13:15.884879+0000', 'last_undegraded': '2024-09-05T13:13:15.884879+0000', 'last_fullsized': '2024-09-05T13:13:15.884879+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 4442, 'log_dups_size': 0, 'ondisk_log_size': 4442, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18169856, 'num_objects': 4436, 'num_object_clones': 0, 'num_object_copies': 8872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 761, 'num_write_kb': 3044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:13:20.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "40'4458", 'reported_seq': 4581, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:16.903407+0000', 'last_change': '2024-09-05T13:13:15.908158+0000', 'last_active': '2024-09-05T13:13:15.908158+0000', 'last_peered': '2024-09-05T13:13:15.856125+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:09.774387+0000', 'last_became_peered': '2024-09-05T13:13:09.774387+0000', 'last_unstale': '2024-09-05T13:13:16.903407+0000', 'last_undegraded': '2024-09-05T13:13:16.903407+0000', 'last_fullsized': '2024-09-05T13:13:16.903407+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 4458, 'log_dups_size': 0, 'ondisk_log_size': 4458, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18231296, 'num_objects': 4451, 'num_object_clones': 0, 'num_object_copies': 8902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2112, 'num_write_kb': 8448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:20.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:13:20.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "40'4462", 'reported_seq': 4586, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:15.889399+0000', 'last_change': '2024-09-05T13:13:15.906584+0000', 'last_active': '2024-09-05T13:13:15.906584+0000', 'last_peered': '2024-09-05T13:13:15.889399+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:09.773402+0000', 'last_became_peered': '2024-09-05T13:13:09.773402+0000', 'last_unstale': '2024-09-05T13:13:15.889399+0000', 'last_undegraded': '2024-09-05T13:13:15.889399+0000', 'last_fullsized': '2024-09-05T13:13:15.889399+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 4462, 'log_dups_size': 0, 'ondisk_log_size': 4462, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18243584, 'num_objects': 4454, 'num_object_clones': 0, 'num_object_copies': 8908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4454, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 829, 'num_write_kb': 3316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.2 is not active+clean 2024-09-05T13:13:20.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.2', 'version': "14'2", 'reported_seq': 88, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:13.777681+0000', 'last_change': '2024-09-05T13:13:15.907337+0000', 'last_active': '2024-09-05T13:13:15.907337+0000', 'last_peered': '2024-09-05T13:13:13.777681+0000', 'last_clean': '2024-09-05T13:13:13.777681+0000', 'last_became_active': '2024-09-05T13:13:09.770617+0000', 'last_became_peered': '2024-09-05T13:13:09.770617+0000', 'last_unstale': '2024-09-05T13:13:13.777681+0000', 'last_undegraded': '2024-09-05T13:13:13.777681+0000', 'last_fullsized': '2024-09-05T13:13:13.777681+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 11, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:28.526981+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:28.526981+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:28.526981+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000597307, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:13:20.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "40'4392", 'reported_seq': 4496, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:16.905660+0000', 'last_change': '2024-09-05T13:13:15.910392+0000', 'last_active': '2024-09-05T13:13:15.910392+0000', 'last_peered': '2024-09-05T13:13:15.895164+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:09.754496+0000', 'last_became_peered': '2024-09-05T13:13:09.754496+0000', 'last_unstale': '2024-09-05T13:13:16.905660+0000', 'last_undegraded': '2024-09-05T13:13:16.905660+0000', 'last_fullsized': '2024-09-05T13:13:16.905660+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 4392, 'log_dups_size': 0, 'ondisk_log_size': 4392, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17969152, 'num_objects': 4387, 'num_object_clones': 0, 'num_object_copies': 8774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4387, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1987, 'num_write_kb': 7948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:20.540 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:13:20.541 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "40'4393", 'reported_seq': 4504, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:16.905522+0000', 'last_change': '2024-09-05T13:13:15.909438+0000', 'last_active': '2024-09-05T13:13:15.909438+0000', 'last_peered': '2024-09-05T13:13:15.890176+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:09.754566+0000', 'last_became_peered': '2024-09-05T13:13:09.754566+0000', 'last_unstale': '2024-09-05T13:13:16.905522+0000', 'last_undegraded': '2024-09-05T13:13:16.905522+0000', 'last_fullsized': '2024-09-05T13:13:16.905522+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 4393, 'log_dups_size': 0, 'ondisk_log_size': 4393, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17969152, 'num_objects': 4387, 'num_object_clones': 0, 'num_object_copies': 8774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4387, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1902, 'num_write_kb': 7608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:20.541 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:13:20.541 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "40'4421", 'reported_seq': 4548, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:15.895786+0000', 'last_change': '2024-09-05T13:13:15.916262+0000', 'last_active': '2024-09-05T13:13:15.916262+0000', 'last_peered': '2024-09-05T13:13:15.895786+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:09.764049+0000', 'last_became_peered': '2024-09-05T13:13:09.764049+0000', 'last_unstale': '2024-09-05T13:13:15.895786+0000', 'last_undegraded': '2024-09-05T13:13:15.895786+0000', 'last_fullsized': '2024-09-05T13:13:15.895786+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 4421, 'log_dups_size': 0, 'ondisk_log_size': 4421, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18100224, 'num_objects': 4419, 'num_object_clones': 0, 'num_object_copies': 8838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 572, 'num_write_kb': 2288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.541 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:13:20.542 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "40'4452", 'reported_seq': 4626, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:15.904984+0000', 'last_change': '2024-09-05T13:13:15.906706+0000', 'last_active': '2024-09-05T13:13:15.906706+0000', 'last_peered': '2024-09-05T13:13:15.904984+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:09.764187+0000', 'last_became_peered': '2024-09-05T13:13:09.764187+0000', 'last_unstale': '2024-09-05T13:13:15.904984+0000', 'last_undegraded': '2024-09-05T13:13:15.904984+0000', 'last_fullsized': '2024-09-05T13:13:15.904984+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 4452, 'log_dups_size': 0, 'ondisk_log_size': 4452, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18202624, 'num_objects': 4444, 'num_object_clones': 0, 'num_object_copies': 8888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 351, 'num_write_kb': 1404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.542 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:13:20.542 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "40'4570", 'reported_seq': 4695, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:15.888158+0000', 'last_change': '2024-09-05T13:13:15.906178+0000', 'last_active': '2024-09-05T13:13:15.906178+0000', 'last_peered': '2024-09-05T13:13:15.888158+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:09.758010+0000', 'last_became_peered': '2024-09-05T13:13:09.758010+0000', 'last_unstale': '2024-09-05T13:13:15.888158+0000', 'last_undegraded': '2024-09-05T13:13:15.888158+0000', 'last_fullsized': '2024-09-05T13:13:15.888158+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 4570, 'log_dups_size': 0, 'ondisk_log_size': 4570, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18702336, 'num_objects': 4566, 'num_object_clones': 0, 'num_object_copies': 9132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 700, 'num_write_kb': 2800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.542 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:13:20.542 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "40'4577", 'reported_seq': 4703, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-05T13:13:15.904752+0000', 'last_change': '2024-09-05T13:13:15.908907+0000', 'last_active': '2024-09-05T13:13:15.908907+0000', 'last_peered': '2024-09-05T13:13:15.904752+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:09.764596+0000', 'last_became_peered': '2024-09-05T13:13:09.764596+0000', 'last_unstale': '2024-09-05T13:13:15.904752+0000', 'last_undegraded': '2024-09-05T13:13:15.904752+0000', 'last_fullsized': '2024-09-05T13:13:15.904752+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 4577, 'log_dups_size': 0, 'ondisk_log_size': 4577, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18698723, 'num_objects': 4566, 'num_object_clones': 0, 'num_object_copies': 9132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1245, 'num_write_kb': 4980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:20.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:13:21.191 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 48 20 72804 72784 5.92239 2.25781 0.0393553 0.0131811 2024-09-05T13:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:21.238 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":128,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"remapped+peering","count":13},{"state_name":"active+clean","count":11}],"num_pgs":24,"num_pools":2,"num_objects":71270,"data_bytes":291910134,"bytes_used":933044224,"bytes_avail":288977248256,"bytes_total":289910292480,"inactive_pgs_ratio":0.54166668653488159,"write_bytes_sec":807790,"read_op_per_sec":0,"write_op_per_sec":196},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:13:21.267 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-05T13:13:21.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:21.838 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:21.872 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-05T13:13:21.523985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":295870966,"num_objects":72237,"num_object_clones":0,"num_object_copies":144474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28036,"num_write_kb":112138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":12307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72319,"ondisk_log_size":72319,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":937752,"kb_used_data":576152,"kb_used_omap":12,"kb_used_meta":361523,"kb_avail":282177768,"statfs":{"total":289910292480,"available":288950034432,"internally_reserved":0,"allocated":589979648,"data_stored":589162577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":370199558},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,6,5,2,0,1,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25381463,"num_objects":6194,"num_object_clones":0,"num_object_copies":12388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35771,"num_objects_misplaced":-49540,"num_objects_unfound":0,"num_objects_dirty":6194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2099,"num_write_kb":8412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004440"},"pg_stats":[{"pgid":"2.f","version":"43'4514","reported_seq":4583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.207667+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:21.207667+0000","last_peered":"2024-09-05T13:13:21.207667+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:21.207667+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18489344,"num_objects":4514,"num_object_clones":0,"num_object_copies":9028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":93},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'4453","reported_seq":4550,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.909806+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:16.909806+0000","last_peered":"2024-09-05T13:13:16.909806+0000","last_clean":"2024-09-05T13:13:16.909806+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:16.909806+0000","last_undegraded":"2024-09-05T13:13:16.909806+0000","last_fullsized":"2024-09-05T13:13:16.909806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4453,"log_dups_size":0,"ondisk_log_size":4453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18214912,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4496","reported_seq":4612,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.903012+0000","last_change":"2024-09-05T13:13:15.910044+0000","last_active":"2024-09-05T13:13:15.910044+0000","last_peered":"2024-09-05T13:13:15.874937+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:09.757821+0000","last_became_peered":"2024-09-05T13:13:09.757821+0000","last_unstale":"2024-09-05T13:13:16.903012+0000","last_undegraded":"2024-09-05T13:13:16.903012+0000","last_fullsized":"2024-09-05T13:13:16.903012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4496,"log_dups_size":0,"ondisk_log_size":4496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18399232,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2307,"num_write_kb":9228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'4442","reported_seq":4504,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.904207+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:16.904207+0000","last_peered":"2024-09-05T13:13:16.904207+0000","last_clean":"2024-09-05T13:13:16.904207+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:16.904207+0000","last_undegraded":"2024-09-05T13:13:16.904207+0000","last_fullsized":"2024-09-05T13:13:16.904207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4442,"log_dups_size":0,"ondisk_log_size":4442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18194432,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4527","reported_seq":4656,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:21.290182+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:21.290182+0000","last_peered":"2024-09-05T13:13:21.290182+0000","last_clean":"2024-09-05T13:13:21.290182+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:21.290182+0000","last_undegraded":"2024-09-05T13:13:21.290182+0000","last_fullsized":"2024-09-05T13:13:21.290182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4527,"log_dups_size":0,"ondisk_log_size":4527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18513920,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1067,"num_write_kb":4268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4562","reported_seq":4694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.225217+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:21.225217+0000","last_peered":"2024-09-05T13:13:21.225217+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:21.225217+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4562,"log_dups_size":0,"ondisk_log_size":4562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1017,"num_write_kb":4068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":106},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4547","reported_seq":4681,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.199743+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:21.199743+0000","last_peered":"2024-09-05T13:13:21.199743+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:21.199743+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4547,"log_dups_size":0,"ondisk_log_size":4547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18599936,"num_objects":4541,"num_object_clones":0,"num_object_copies":9082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":866,"num_write_kb":3464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":105},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4575","reported_seq":4701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:21.237848+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:21.237848+0000","last_peered":"2024-09-05T13:13:21.237848+0000","last_clean":"2024-09-05T13:13:21.237848+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:21.237848+0000","last_undegraded":"2024-09-05T13:13:21.237848+0000","last_fullsized":"2024-09-05T13:13:21.237848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1185,"num_write_kb":4740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"40'4458","reported_seq":4581,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.903407+0000","last_change":"2024-09-05T13:13:15.908158+0000","last_active":"2024-09-05T13:13:15.908158+0000","last_peered":"2024-09-05T13:13:15.856125+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:09.774387+0000","last_became_peered":"2024-09-05T13:13:09.774387+0000","last_unstale":"2024-09-05T13:13:16.903407+0000","last_undegraded":"2024-09-05T13:13:16.903407+0000","last_fullsized":"2024-09-05T13:13:16.903407+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18231296,"num_objects":4451,"num_object_clones":0,"num_object_copies":8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":8448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.910327+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:16.910327+0000","last_peered":"2024-09-05T13:13:16.910327+0000","last_clean":"2024-09-05T13:13:16.910327+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:16.910327+0000","last_undegraded":"2024-09-05T13:13:16.910327+0000","last_fullsized":"2024-09-05T13:13:16.910327+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4561","reported_seq":4691,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.282611+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:21.282611+0000","last_peered":"2024-09-05T13:13:21.282611+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:21.282611+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4561,"log_dups_size":0,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18649088,"num_objects":4553,"num_object_clones":0,"num_object_copies":9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4392","reported_seq":4496,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.905660+0000","last_change":"2024-09-05T13:13:15.910392+0000","last_active":"2024-09-05T13:13:15.910392+0000","last_peered":"2024-09-05T13:13:15.895164+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:09.754496+0000","last_became_peered":"2024-09-05T13:13:09.754496+0000","last_unstale":"2024-09-05T13:13:16.905660+0000","last_undegraded":"2024-09-05T13:13:16.905660+0000","last_fullsized":"2024-09-05T13:13:16.905660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4392,"log_dups_size":0,"ondisk_log_size":4392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1987,"num_write_kb":7948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"40'4393","reported_seq":4504,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-05T13:13:16.905522+0000","last_change":"2024-09-05T13:13:15.909438+0000","last_active":"2024-09-05T13:13:15.909438+0000","last_peered":"2024-09-05T13:13:15.890176+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:09.754566+0000","last_became_peered":"2024-09-05T13:13:09.754566+0000","last_unstale":"2024-09-05T13:13:16.905522+0000","last_undegraded":"2024-09-05T13:13:16.905522+0000","last_fullsized":"2024-09-05T13:13:16.905522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4521","reported_seq":4655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.282995+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:21.282995+0000","last_peered":"2024-09-05T13:13:21.282995+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:21.282995+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18509824,"num_objects":4519,"num_object_clones":0,"num_object_copies":9038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":100},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4550","reported_seq":4732,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.240283+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:21.240283+0000","last_peered":"2024-09-05T13:13:21.240283+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:21.240283+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":449,"num_write_kb":1796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":98},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-05T13:13:16.910577+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:16.910577+0000","last_peered":"2024-09-05T13:13:16.910577+0000","last_clean":"2024-09-05T13:13:16.910577+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:16.910577+0000","last_undegraded":"2024-09-05T13:13:16.910577+0000","last_fullsized":"2024-09-05T13:13:16.910577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4661","reported_seq":4794,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.218907+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:21.218907+0000","last_peered":"2024-09-05T13:13:21.218907+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:21.218907+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":791,"num_write_kb":3164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":91},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4664","reported_seq":4798,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.231753+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:21.231753+0000","last_peered":"2024-09-05T13:13:21.231753+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:21.231753+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19055075,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1332,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":88},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":295870947,"num_objects":72235,"num_object_clones":0,"num_object_copies":144470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28034,"num_write_kb":112136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":12288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589312000,"data_stored":589221675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72316,"ondisk_log_size":72316,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425392,"kb_used_data":285404,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93946448,"statfs":{"total":96636764160,"available":96201162752,"internally_reserved":0,"allocated":292253696,"data_stored":292014445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":143322797},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447744,"kb_used_data":289776,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93924096,"statfs":{"total":96636764160,"available":96178274304,"internally_reserved":0,"allocated":296730624,"data_stored":296481457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64616,"kb_used_data":972,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94307224,"statfs":{"total":96636764160,"available":96570597376,"internally_reserved":0,"allocated":995328,"data_stored":666675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":296738816,"data_stored":296735203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":584549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":291905536,"data_stored":291901923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:22.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 49 20 73404 73384 5.84921 2.34375 0.0215915 0.0133476 2024-09-05T13:13:23.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 50 20 74057 74037 5.78324 2.55078 0.0521704 0.0135017 2024-09-05T13:13:24.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 51 20 74691 74671 5.71841 2.47656 0.0852934 0.0136513 2024-09-05T13:13:24.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:25.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 52 20 75322 75302 5.65584 2.46484 0.0370955 0.0138052 2024-09-05T13:13:25.452 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:25.483 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-05T13:13:23.524416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":299287030,"num_objects":73071,"num_object_clones":0,"num_object_copies":146142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":73071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28870,"num_write_kb":115474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":274451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73153,"ondisk_log_size":73153,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":964300,"kb_used_data":580496,"kb_used_omap":12,"kb_used_meta":383731,"kb_avail":282151220,"statfs":{"total":289910292480,"available":288922849280,"internally_reserved":0,"allocated":594427904,"data_stored":593604994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":392940550},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,4,1,2,0,1,0,0,8,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28797527,"num_objects":7028,"num_object_clones":0,"num_object_copies":14056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":-45219,"num_objects_unfound":0,"num_objects_dirty":7028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2933,"num_write_kb":11748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.627988"},"pg_stats":[{"pgid":"2.f","version":"43'4514","reported_seq":4583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.207667+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:21.207667+0000","last_peered":"2024-09-05T13:13:21.207667+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:21.207667+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18489344,"num_objects":4514,"num_object_clones":0,"num_object_copies":9028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":93},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4590","reported_seq":4689,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:22.155085+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:22.155085+0000","last_peered":"2024-09-05T13:13:22.155085+0000","last_clean":"2024-09-05T13:13:22.155085+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:22.155085+0000","last_undegraded":"2024-09-05T13:13:22.155085+0000","last_fullsized":"2024-09-05T13:13:22.155085+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2600,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4640","reported_seq":4762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.174977+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:22.174977+0000","last_peered":"2024-09-05T13:13:22.174977+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:22.174977+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2451,"num_write_kb":9804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4590","reported_seq":4654,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:22.132038+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:22.132038+0000","last_peered":"2024-09-05T13:13:22.132038+0000","last_clean":"2024-09-05T13:13:22.132038+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:22.132038+0000","last_undegraded":"2024-09-05T13:13:22.132038+0000","last_fullsized":"2024-09-05T13:13:22.132038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4590,"num_write_kb":18360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4527","reported_seq":4656,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:21.290182+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:21.290182+0000","last_peered":"2024-09-05T13:13:21.290182+0000","last_clean":"2024-09-05T13:13:21.290182+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:21.290182+0000","last_undegraded":"2024-09-05T13:13:21.290182+0000","last_fullsized":"2024-09-05T13:13:21.290182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4527,"log_dups_size":0,"ondisk_log_size":4527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18513920,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1067,"num_write_kb":4268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4562","reported_seq":4694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.225217+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:21.225217+0000","last_peered":"2024-09-05T13:13:21.225217+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:21.225217+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4562,"log_dups_size":0,"ondisk_log_size":4562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1017,"num_write_kb":4068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":106},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4547","reported_seq":4681,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.199743+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:21.199743+0000","last_peered":"2024-09-05T13:13:21.199743+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:21.199743+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4547,"log_dups_size":0,"ondisk_log_size":4547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18599936,"num_objects":4541,"num_object_clones":0,"num_object_copies":9082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":866,"num_write_kb":3464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":105},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4575","reported_seq":4701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:21.237848+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:21.237848+0000","last_peered":"2024-09-05T13:13:21.237848+0000","last_clean":"2024-09-05T13:13:21.237848+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:21.237848+0000","last_undegraded":"2024-09-05T13:13:21.237848+0000","last_fullsized":"2024-09-05T13:13:21.237848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1185,"num_write_kb":4740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4602","reported_seq":4731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.172377+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:22.172377+0000","last_peered":"2024-09-05T13:13:22.172377+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:22.172377+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821120,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4561","reported_seq":4691,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.282611+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:21.282611+0000","last_peered":"2024-09-05T13:13:21.282611+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:21.282611+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4561,"log_dups_size":0,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18649088,"num_objects":4553,"num_object_clones":0,"num_object_copies":9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4535","reported_seq":4772,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.181697+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:22.181697+0000","last_peered":"2024-09-05T13:13:22.181697+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:22.181697+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4535,"log_dups_size":0,"ondisk_log_size":4535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18554880,"num_objects":4530,"num_object_clones":0,"num_object_copies":9060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":4530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2130,"num_write_kb":8520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4321},{"shards":"2","objects":207}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4511","reported_seq":4628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.164713+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:22.164713+0000","last_peered":"2024-09-05T13:13:22.164713+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:22.164713+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4511,"log_dups_size":0,"ondisk_log_size":4511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2020,"num_write_kb":8080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":118}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4521","reported_seq":4655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.282995+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:21.282995+0000","last_peered":"2024-09-05T13:13:21.282995+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:21.282995+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18509824,"num_objects":4519,"num_object_clones":0,"num_object_copies":9038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":100},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4550","reported_seq":4732,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.240283+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:21.240283+0000","last_peered":"2024-09-05T13:13:21.240283+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:21.240283+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":449,"num_write_kb":1796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":98},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4661","reported_seq":4794,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.218907+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:21.218907+0000","last_peered":"2024-09-05T13:13:21.218907+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:21.218907+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":791,"num_write_kb":3164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":91},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4664","reported_seq":4798,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.231753+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:21.231753+0000","last_peered":"2024-09-05T13:13:21.231753+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:21.231753+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19055075,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1332,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":88},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":299287011,"num_objects":73069,"num_object_clones":0,"num_object_copies":146138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":73069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28868,"num_write_kb":115472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":274432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593813504,"data_stored":593723179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73150,"ondisk_log_size":73150,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451940,"kb_used_data":289748,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93919900,"statfs":{"total":96636764160,"available":96173977600,"internally_reserved":0,"allocated":296701952,"data_stored":296456862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":166063789},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,0,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447744,"kb_used_data":289776,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93924096,"statfs":{"total":96636764160,"available":96178274304,"internally_reserved":0,"allocated":296730624,"data_stored":296481457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64616,"kb_used_data":972,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94307224,"statfs":{"total":96636764160,"available":96570597376,"internally_reserved":0,"allocated":995328,"data_stored":666675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":296738816,"data_stored":296735203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":584549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296407040,"data_stored":296403427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:25.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:26.022 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:26.050 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-05T13:13:25.524827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":299287030,"num_objects":73071,"num_object_clones":0,"num_object_copies":146142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":73071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28870,"num_write_kb":115474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":274451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73153,"ondisk_log_size":73153,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":980164,"kb_used_data":588048,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282135356,"statfs":{"total":289910292480,"available":288906604544,"internally_reserved":0,"allocated":602161152,"data_stored":601338242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":401460230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,3,3,1,0,1,0,0,8,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14397440,"num_objects":3515,"num_object_clones":0,"num_object_copies":7030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":-12285,"num_objects_unfound":0,"num_objects_dirty":3515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3515,"num_write_kb":14060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.626486"},"pg_stats":[{"pgid":"2.f","version":"43'4514","reported_seq":4583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.207667+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:21.207667+0000","last_peered":"2024-09-05T13:13:21.207667+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:21.207667+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18489344,"num_objects":4514,"num_object_clones":0,"num_object_copies":9028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":93},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4590","reported_seq":4689,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:22.155085+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:22.155085+0000","last_peered":"2024-09-05T13:13:22.155085+0000","last_clean":"2024-09-05T13:13:22.155085+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:22.155085+0000","last_undegraded":"2024-09-05T13:13:22.155085+0000","last_fullsized":"2024-09-05T13:13:22.155085+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2600,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4640","reported_seq":4762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.174977+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:22.174977+0000","last_peered":"2024-09-05T13:13:22.174977+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:22.174977+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2451,"num_write_kb":9804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4590","reported_seq":4654,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:22.132038+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:22.132038+0000","last_peered":"2024-09-05T13:13:22.132038+0000","last_clean":"2024-09-05T13:13:22.132038+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:22.132038+0000","last_undegraded":"2024-09-05T13:13:22.132038+0000","last_fullsized":"2024-09-05T13:13:22.132038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4590,"num_write_kb":18360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4527","reported_seq":4656,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:21.290182+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:21.290182+0000","last_peered":"2024-09-05T13:13:21.290182+0000","last_clean":"2024-09-05T13:13:21.290182+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:21.290182+0000","last_undegraded":"2024-09-05T13:13:21.290182+0000","last_fullsized":"2024-09-05T13:13:21.290182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4527,"log_dups_size":0,"ondisk_log_size":4527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18513920,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1067,"num_write_kb":4268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4562","reported_seq":4694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.225217+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:21.225217+0000","last_peered":"2024-09-05T13:13:21.225217+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:21.225217+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4562,"log_dups_size":0,"ondisk_log_size":4562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1017,"num_write_kb":4068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":106},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4547","reported_seq":4681,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.199743+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:21.199743+0000","last_peered":"2024-09-05T13:13:21.199743+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:21.199743+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4547,"log_dups_size":0,"ondisk_log_size":4547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18599936,"num_objects":4541,"num_object_clones":0,"num_object_copies":9082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":866,"num_write_kb":3464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":105},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4575","reported_seq":4701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:21.237848+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:21.237848+0000","last_peered":"2024-09-05T13:13:21.237848+0000","last_clean":"2024-09-05T13:13:21.237848+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:21.237848+0000","last_undegraded":"2024-09-05T13:13:21.237848+0000","last_fullsized":"2024-09-05T13:13:21.237848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1185,"num_write_kb":4740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4602","reported_seq":4731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.172377+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:22.172377+0000","last_peered":"2024-09-05T13:13:22.172377+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:22.172377+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821120,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4561","reported_seq":4691,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.282611+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:21.282611+0000","last_peered":"2024-09-05T13:13:21.282611+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:21.282611+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4561,"log_dups_size":0,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18649088,"num_objects":4553,"num_object_clones":0,"num_object_copies":9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4535","reported_seq":4772,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.181697+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:22.181697+0000","last_peered":"2024-09-05T13:13:22.181697+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:22.181697+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4535,"log_dups_size":0,"ondisk_log_size":4535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18554880,"num_objects":4530,"num_object_clones":0,"num_object_copies":9060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":4530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2130,"num_write_kb":8520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4321},{"shards":"2","objects":207}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4511","reported_seq":4628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.164713+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:22.164713+0000","last_peered":"2024-09-05T13:13:22.164713+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:22.164713+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4511,"log_dups_size":0,"ondisk_log_size":4511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2020,"num_write_kb":8080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":118}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4521","reported_seq":4655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.282995+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:21.282995+0000","last_peered":"2024-09-05T13:13:21.282995+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:21.282995+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18509824,"num_objects":4519,"num_object_clones":0,"num_object_copies":9038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":100},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4550","reported_seq":4732,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.240283+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:21.240283+0000","last_peered":"2024-09-05T13:13:21.240283+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:21.240283+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":449,"num_write_kb":1796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":98},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4661","reported_seq":4794,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.218907+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:21.218907+0000","last_peered":"2024-09-05T13:13:21.218907+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:21.218907+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":791,"num_write_kb":3164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":91},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4664","reported_seq":4798,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.231753+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:21.231753+0000","last_peered":"2024-09-05T13:13:21.231753+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:21.231753+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19055075,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1332,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":88},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":299287011,"num_objects":73069,"num_object_clones":0,"num_object_copies":146138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":73069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28868,"num_write_kb":115472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":274432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":603430912,"data_stored":603340587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73150,"ondisk_log_size":73150,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451940,"kb_used_data":289748,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93919900,"statfs":{"total":96636764160,"available":96173977600,"internally_reserved":0,"allocated":296701952,"data_stored":296456862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":166063789},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,0,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447744,"kb_used_data":289776,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93924096,"statfs":{"total":96636764160,"available":96178274304,"internally_reserved":0,"allocated":296730624,"data_stored":296481457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80480,"kb_used_data":8524,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94291360,"statfs":{"total":96636764160,"available":96554352640,"internally_reserved":0,"allocated":8728576,"data_stored":8399923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":73658031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":296738816,"data_stored":296735203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10285056,"data_stored":10201957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296407040,"data_stored":296403427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:26.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:13:26.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4514", 'reported_seq': 4583, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:21.207667+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:13:21.207667+0000', 'last_peered': '2024-09-05T13:13:21.207667+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:13:21.207667+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 4514, 'log_dups_size': 0, 'ondisk_log_size': 4514, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18489344, 'num_objects': 4514, 'num_object_clones': 0, 'num_object_copies': 9028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4514, 'num_write_kb': 18056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 93}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:26.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:13:26.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4640", 'reported_seq': 4762, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:22.174977+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:13:22.174977+0000', 'last_peered': '2024-09-05T13:13:22.174977+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:13:22.174977+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 4640, 'log_dups_size': 0, 'ondisk_log_size': 4640, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18989056, 'num_objects': 4636, 'num_object_clones': 0, 'num_object_copies': 9272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2451, 'num_write_kb': 9804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:26.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4562", 'reported_seq': 4694, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:21.225217+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:13:21.225217+0000', 'last_peered': '2024-09-05T13:13:21.225217+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:13:21.225217+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 4562, 'log_dups_size': 0, 'ondisk_log_size': 4562, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18665472, 'num_objects': 4557, 'num_object_clones': 0, 'num_object_copies': 9114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4557, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1017, 'num_write_kb': 4068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 106}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4547", 'reported_seq': 4681, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:21.199743+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:13:21.199743+0000', 'last_peered': '2024-09-05T13:13:21.199743+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:13:21.199743+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 4547, 'log_dups_size': 0, 'ondisk_log_size': 4547, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18599936, 'num_objects': 4541, 'num_object_clones': 0, 'num_object_copies': 9082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 866, 'num_write_kb': 3464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 105}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4602", 'reported_seq': 4731, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:22.172377+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:13:22.172377+0000', 'last_peered': '2024-09-05T13:13:22.172377+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:13:22.172377+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 4602, 'log_dups_size': 0, 'ondisk_log_size': 4602, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18821120, 'num_objects': 4595, 'num_object_clones': 0, 'num_object_copies': 9190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4595, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2256, 'num_write_kb': 9024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4561", 'reported_seq': 4691, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:21.282611+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:13:21.282611+0000', 'last_peered': '2024-09-05T13:13:21.282611+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:13:21.282611+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 4561, 'log_dups_size': 0, 'ondisk_log_size': 4561, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18649088, 'num_objects': 4553, 'num_object_clones': 0, 'num_object_copies': 9106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4553, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 928, 'num_write_kb': 3712, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 99}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4535", 'reported_seq': 4772, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:22.181697+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:13:22.181697+0000', 'last_peered': '2024-09-05T13:13:22.181697+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:13:22.181697+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 4535, 'log_dups_size': 0, 'ondisk_log_size': 4535, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18554880, 'num_objects': 4530, 'num_object_clones': 0, 'num_object_copies': 9060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 4321, 'num_objects_unfound': 0, 'num_objects_dirty': 4530, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2130, 'num_write_kb': 8520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 64, 'num_bytes_recovered': 262144, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4321}, {'shards': '2', 'objects': 207}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:26.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4511", 'reported_seq': 4628, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:22.164713+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:13:22.164713+0000', 'last_peered': '2024-09-05T13:13:22.164713+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:13:22.164713+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 4511, 'log_dups_size': 0, 'ondisk_log_size': 4511, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18452480, 'num_objects': 4505, 'num_object_clones': 0, 'num_object_copies': 9010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2020, 'num_write_kb': 8080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 118}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4521", 'reported_seq': 4655, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:21.282995+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:13:21.282995+0000', 'last_peered': '2024-09-05T13:13:21.282995+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:13:21.282995+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 4521, 'log_dups_size': 0, 'ondisk_log_size': 4521, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18509824, 'num_objects': 4519, 'num_object_clones': 0, 'num_object_copies': 9038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4519, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 672, 'num_write_kb': 2688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 100}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4550", 'reported_seq': 4732, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:21.240283+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:13:21.240283+0000', 'last_peered': '2024-09-05T13:13:21.240283+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:13:21.240283+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 4550, 'log_dups_size': 0, 'ondisk_log_size': 4550, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18604032, 'num_objects': 4542, 'num_object_clones': 0, 'num_object_copies': 9084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4542, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 449, 'num_write_kb': 1796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 98}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4661", 'reported_seq': 4794, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:21.218907+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:13:21.218907+0000', 'last_peered': '2024-09-05T13:13:21.218907+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:13:21.218907+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 4661, 'log_dups_size': 0, 'ondisk_log_size': 4661, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19075072, 'num_objects': 4657, 'num_object_clones': 0, 'num_object_copies': 9314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 791, 'num_write_kb': 3164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 91}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:13:26.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4664", 'reported_seq': 4798, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:21.231753+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:13:21.231753+0000', 'last_peered': '2024-09-05T13:13:21.231753+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:13:21.231753+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 4664, 'log_dups_size': 0, 'ondisk_log_size': 4664, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19055075, 'num_objects': 4653, 'num_object_clones': 0, 'num_object_copies': 9306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4653, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1332, 'num_write_kb': 5328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 88}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:26.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:13:26.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 53 20 75871 75851 5.58959 2.14453 0.033596 0.0139661 2024-09-05T13:13:26.740 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:26.740 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/146142 objects degraded (33.603%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":133,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":73071,"data_bytes":299287030,"bytes_used":1003687936,"bytes_avail":288906604544,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":146142,"degraded_ratio":0.33602934132556006,"misplaced_objects":4321,"misplaced_total":146142,"misplaced_ratio":0.029567133336070396,"recovering_objects_per_sec":7,"recovering_bytes_per_sec":30388,"recovering_keys_per_sec":0,"num_objects_recovered":64,"num_bytes_recovered":262144,"num_keys_recovered":0,"write_bytes_sec":1668980,"read_op_per_sec":0,"write_op_per_sec":407},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:13:26.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:13:26.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:27.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 54 19 76424 76405 5.52616 2.16406 0.0315 0.0141285 2024-09-05T13:13:27.318 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:27.319 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:27.353 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-05T13:13:25.524827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":299287030,"num_objects":73071,"num_object_clones":0,"num_object_copies":146142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":73071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28870,"num_write_kb":115474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":274451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73153,"ondisk_log_size":73153,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":980164,"kb_used_data":588048,"kb_used_omap":12,"kb_used_meta":392051,"kb_avail":282135356,"statfs":{"total":289910292480,"available":288906604544,"internally_reserved":0,"allocated":602161152,"data_stored":601338242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":401460230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,3,3,1,0,1,0,0,8,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14397440,"num_objects":3515,"num_object_clones":0,"num_object_copies":7030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":-12285,"num_objects_unfound":0,"num_objects_dirty":3515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3515,"num_write_kb":14060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.626486"},"pg_stats":[{"pgid":"2.f","version":"43'4514","reported_seq":4583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.207667+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:21.207667+0000","last_peered":"2024-09-05T13:13:21.207667+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:21.207667+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18489344,"num_objects":4514,"num_object_clones":0,"num_object_copies":9028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":93},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4590","reported_seq":4689,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:22.155085+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:22.155085+0000","last_peered":"2024-09-05T13:13:22.155085+0000","last_clean":"2024-09-05T13:13:22.155085+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:22.155085+0000","last_undegraded":"2024-09-05T13:13:22.155085+0000","last_fullsized":"2024-09-05T13:13:22.155085+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18776064,"num_objects":4584,"num_object_clones":0,"num_object_copies":9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2600,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4640","reported_seq":4762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.174977+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:22.174977+0000","last_peered":"2024-09-05T13:13:22.174977+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:22.174977+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2451,"num_write_kb":9804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4590","reported_seq":4654,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:22.132038+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:22.132038+0000","last_peered":"2024-09-05T13:13:22.132038+0000","last_clean":"2024-09-05T13:13:22.132038+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:22.132038+0000","last_undegraded":"2024-09-05T13:13:22.132038+0000","last_fullsized":"2024-09-05T13:13:22.132038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4590,"num_write_kb":18360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4527","reported_seq":4656,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:21.290182+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:21.290182+0000","last_peered":"2024-09-05T13:13:21.290182+0000","last_clean":"2024-09-05T13:13:21.290182+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:21.290182+0000","last_undegraded":"2024-09-05T13:13:21.290182+0000","last_fullsized":"2024-09-05T13:13:21.290182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4527,"log_dups_size":0,"ondisk_log_size":4527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18513920,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1067,"num_write_kb":4268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4562","reported_seq":4694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.225217+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:21.225217+0000","last_peered":"2024-09-05T13:13:21.225217+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:21.225217+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4562,"log_dups_size":0,"ondisk_log_size":4562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18665472,"num_objects":4557,"num_object_clones":0,"num_object_copies":9114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1017,"num_write_kb":4068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":106},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4547","reported_seq":4681,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.199743+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:21.199743+0000","last_peered":"2024-09-05T13:13:21.199743+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:21.199743+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4547,"log_dups_size":0,"ondisk_log_size":4547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18599936,"num_objects":4541,"num_object_clones":0,"num_object_copies":9082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":866,"num_write_kb":3464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":105},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4575","reported_seq":4701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:21.237848+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:21.237848+0000","last_peered":"2024-09-05T13:13:21.237848+0000","last_clean":"2024-09-05T13:13:21.237848+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:21.237848+0000","last_undegraded":"2024-09-05T13:13:21.237848+0000","last_fullsized":"2024-09-05T13:13:21.237848+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1185,"num_write_kb":4740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4602","reported_seq":4731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.172377+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:22.172377+0000","last_peered":"2024-09-05T13:13:22.172377+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:22.172377+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821120,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":144}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4561","reported_seq":4691,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.282611+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:21.282611+0000","last_peered":"2024-09-05T13:13:21.282611+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:21.282611+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4561,"log_dups_size":0,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18649088,"num_objects":4553,"num_object_clones":0,"num_object_copies":9106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4535","reported_seq":4772,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.181697+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:22.181697+0000","last_peered":"2024-09-05T13:13:22.181697+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:22.181697+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4535,"log_dups_size":0,"ondisk_log_size":4535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18554880,"num_objects":4530,"num_object_clones":0,"num_object_copies":9060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":4530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2130,"num_write_kb":8520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4321},{"shards":"2","objects":207}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4511","reported_seq":4628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:22.164713+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:22.164713+0000","last_peered":"2024-09-05T13:13:22.164713+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:22.164713+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4511,"log_dups_size":0,"ondisk_log_size":4511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18452480,"num_objects":4505,"num_object_clones":0,"num_object_copies":9010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2020,"num_write_kb":8080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":118}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4521","reported_seq":4655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.282995+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:21.282995+0000","last_peered":"2024-09-05T13:13:21.282995+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:21.282995+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18509824,"num_objects":4519,"num_object_clones":0,"num_object_copies":9038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":2688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":100},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4550","reported_seq":4732,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.240283+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:21.240283+0000","last_peered":"2024-09-05T13:13:21.240283+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:21.240283+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":449,"num_write_kb":1796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":98},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4661","reported_seq":4794,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.218907+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:21.218907+0000","last_peered":"2024-09-05T13:13:21.218907+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:21.218907+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":791,"num_write_kb":3164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":91},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4664","reported_seq":4798,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:21.231753+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:21.231753+0000","last_peered":"2024-09-05T13:13:21.231753+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:21.231753+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19055075,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1332,"num_write_kb":5328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":88},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":299287011,"num_objects":73069,"num_object_clones":0,"num_object_copies":146138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4321,"num_objects_unfound":0,"num_objects_dirty":73069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28868,"num_write_kb":115472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":274432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":603430912,"data_stored":603340587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73150,"ondisk_log_size":73150,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451940,"kb_used_data":289748,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93919900,"statfs":{"total":96636764160,"available":96173977600,"internally_reserved":0,"allocated":296701952,"data_stored":296456862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":166063789},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,0,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447744,"kb_used_data":289776,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93924096,"statfs":{"total":96636764160,"available":96178274304,"internally_reserved":0,"allocated":296730624,"data_stored":296481457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80480,"kb_used_data":8524,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94291360,"statfs":{"total":96636764160,"available":96554352640,"internally_reserved":0,"allocated":8728576,"data_stored":8399923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":73658031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":296738816,"data_stored":296735203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10285056,"data_stored":10201957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":296407040,"data_stored":296403427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:28.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 55 20 77046 77026 5.46979 2.42578 0.0377803 0.0142755 2024-09-05T13:13:29.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 56 20 77669 77649 5.41557 2.43359 0.0593966 0.0144163 2024-09-05T13:13:30.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 57 20 78283 78263 5.36264 2.39844 0.0523782 0.0145598 2024-09-05T13:13:30.355 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:30.920 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:30.947 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-05T13:13:29.525677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":311742966,"num_objects":76112,"num_object_clones":0,"num_object_copies":152224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":76112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31911,"num_write_kb":127638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":692243,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76194,"ondisk_log_size":76194,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1034208,"kb_used_data":611564,"kb_used_omap":12,"kb_used_meta":422579,"kb_avail":282081312,"statfs":{"total":289910292480,"available":288851263488,"internally_reserved":0,"allocated":626241536,"data_stored":625418626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":432720902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,8,1,4,5,16,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19832832,"num_objects":4842,"num_object_clones":0,"num_object_copies":9684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4218,"num_objects_unfound":0,"num_objects_dirty":4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4842,"num_write_kb":19368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.625982"},"pg_stats":[{"pgid":"2.f","version":"43'4696","reported_seq":4765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.292332+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:26.292332+0000","last_peered":"2024-09-05T13:13:26.292332+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:26.292332+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19234816,"num_objects":4696,"num_object_clones":0,"num_object_copies":9392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":18784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":275},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4763","reported_seq":4862,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:27.202815+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:27.202815+0000","last_peered":"2024-09-05T13:13:27.202815+0000","last_clean":"2024-09-05T13:13:27.202815+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:27.202815+0000","last_undegraded":"2024-09-05T13:13:27.202815+0000","last_fullsized":"2024-09-05T13:13:27.202815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4763,"log_dups_size":0,"ondisk_log_size":4763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19484672,"num_objects":4757,"num_object_clones":0,"num_object_copies":9514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4800","reported_seq":4922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.160722+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:27.160722+0000","last_peered":"2024-09-05T13:13:27.160722+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:27.160722+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2611,"num_write_kb":10444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":304}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4796","reported_seq":4860,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:27.151241+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:27.151241+0000","last_peered":"2024-09-05T13:13:27.151241+0000","last_clean":"2024-09-05T13:13:27.151241+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:27.151241+0000","last_undegraded":"2024-09-05T13:13:27.151241+0000","last_fullsized":"2024-09-05T13:13:27.151241+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4796,"num_write_kb":19184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4710","reported_seq":4839,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:26.209163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:26.209163+0000","last_peered":"2024-09-05T13:13:26.209163+0000","last_clean":"2024-09-05T13:13:26.209163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:26.209163+0000","last_undegraded":"2024-09-05T13:13:26.209163+0000","last_fullsized":"2024-09-05T13:13:26.209163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19263488,"num_objects":4703,"num_object_clones":0,"num_object_copies":9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1250,"num_write_kb":5000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4773","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.223092+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:26.223092+0000","last_peered":"2024-09-05T13:13:26.223092+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:26.223092+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4773,"log_dups_size":0,"ondisk_log_size":4773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529728,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1228,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":317},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4708","reported_seq":4840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.265322+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:26.265322+0000","last_peered":"2024-09-05T13:13:26.265322+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:26.265322+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259392,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1027,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":266},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4790","reported_seq":4916,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:26.211113+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:26.211113+0000","last_peered":"2024-09-05T13:13:26.211113+0000","last_clean":"2024-09-05T13:13:26.211113+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:26.211113+0000","last_undegraded":"2024-09-05T13:13:26.211113+0000","last_fullsized":"2024-09-05T13:13:26.211113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4790,"log_dups_size":0,"ondisk_log_size":4790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19611648,"num_objects":4788,"num_object_clones":0,"num_object_copies":9576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4786","reported_seq":4915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.174963+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:27.174963+0000","last_peered":"2024-09-05T13:13:27.174963+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:27.174963+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4786,"log_dups_size":0,"ondisk_log_size":4786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19574784,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":328}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4741","reported_seq":4872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.268809+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:26.268809+0000","last_peered":"2024-09-05T13:13:26.268809+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:26.268809+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19386368,"num_objects":4733,"num_object_clones":0,"num_object_copies":9466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1108,"num_write_kb":4432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":279},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4743","reported_seq":5186,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.114804+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:27.114804+0000","last_peered":"2024-09-05T13:13:27.114804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:27.114804+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19406848,"num_objects":4738,"num_object_clones":0,"num_object_copies":9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4219},{"shards":"2","objects":517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4717","reported_seq":4834,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.168820+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:27.168820+0000","last_peered":"2024-09-05T13:13:27.168820+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:27.168820+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4717,"log_dups_size":0,"ondisk_log_size":4717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4712","reported_seq":4847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.288164+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:26.288164+0000","last_peered":"2024-09-05T13:13:26.288164+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:26.288164+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292160,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":863,"num_write_kb":3452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":291},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4757","reported_seq":4939,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.237618+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:26.237618+0000","last_peered":"2024-09-05T13:13:26.237618+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:26.237618+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4757,"log_dups_size":0,"ondisk_log_size":4757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19451904,"num_objects":4749,"num_object_clones":0,"num_object_copies":9498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":305},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4844","reported_seq":4977,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.294145+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:26.294145+0000","last_peered":"2024-09-05T13:13:26.294145+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:26.294145+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19824640,"num_objects":4840,"num_object_clones":0,"num_object_copies":9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":974,"num_write_kb":3896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":274},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4855","reported_seq":4988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.257137+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:26.257137+0000","last_peered":"2024-09-05T13:13:26.257137+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:26.257137+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4855,"log_dups_size":0,"ondisk_log_size":4855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19837411,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1523,"num_write_kb":6092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":279},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":311742947,"num_objects":76110,"num_object_clones":0,"num_object_copies":152220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":76110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31909,"num_write_kb":127636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":628776960,"data_stored":628686635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76191,"ondisk_log_size":76191,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461212,"kb_used_data":294860,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93910628,"statfs":{"total":96636764160,"available":96164483072,"internally_reserved":0,"allocated":301936640,"data_stored":301691550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":170323629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461032,"kb_used_data":298900,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93910808,"statfs":{"total":96636764160,"available":96164667392,"internally_reserved":0,"allocated":306073600,"data_stored":305824433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,1,1,2,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111964,"kb_used_data":17804,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94259876,"statfs":{"total":96636764160,"available":96522113024,"internally_reserved":0,"allocated":18231296,"data_stored":17902643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306216960,"data_stored":306213347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19886080,"data_stored":19802981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302673920,"data_stored":302670307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:30.948 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:31.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 58 20 78845 78825 5.30803 2.19531 0.0226522 0.0147104 2024-09-05T13:13:31.497 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:31.500 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:31.527 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-05T13:13:29.525677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":311742966,"num_objects":76112,"num_object_clones":0,"num_object_copies":152224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":76112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31911,"num_write_kb":127638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":692243,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76194,"ondisk_log_size":76194,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1034208,"kb_used_data":611564,"kb_used_omap":12,"kb_used_meta":422579,"kb_avail":282081312,"statfs":{"total":289910292480,"available":288851263488,"internally_reserved":0,"allocated":626241536,"data_stored":625418626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":432720902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,8,1,4,5,16,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19832832,"num_objects":4842,"num_object_clones":0,"num_object_copies":9684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4218,"num_objects_unfound":0,"num_objects_dirty":4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4842,"num_write_kb":19368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.625982"},"pg_stats":[{"pgid":"2.f","version":"43'4696","reported_seq":4765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.292332+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:26.292332+0000","last_peered":"2024-09-05T13:13:26.292332+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:26.292332+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19234816,"num_objects":4696,"num_object_clones":0,"num_object_copies":9392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":18784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":275},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4763","reported_seq":4862,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:27.202815+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:27.202815+0000","last_peered":"2024-09-05T13:13:27.202815+0000","last_clean":"2024-09-05T13:13:27.202815+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:27.202815+0000","last_undegraded":"2024-09-05T13:13:27.202815+0000","last_fullsized":"2024-09-05T13:13:27.202815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4763,"log_dups_size":0,"ondisk_log_size":4763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19484672,"num_objects":4757,"num_object_clones":0,"num_object_copies":9514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4800","reported_seq":4922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.160722+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:27.160722+0000","last_peered":"2024-09-05T13:13:27.160722+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:27.160722+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2611,"num_write_kb":10444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":304}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4796","reported_seq":4860,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:27.151241+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:27.151241+0000","last_peered":"2024-09-05T13:13:27.151241+0000","last_clean":"2024-09-05T13:13:27.151241+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:27.151241+0000","last_undegraded":"2024-09-05T13:13:27.151241+0000","last_fullsized":"2024-09-05T13:13:27.151241+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4796,"num_write_kb":19184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4710","reported_seq":4839,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:26.209163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:26.209163+0000","last_peered":"2024-09-05T13:13:26.209163+0000","last_clean":"2024-09-05T13:13:26.209163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:26.209163+0000","last_undegraded":"2024-09-05T13:13:26.209163+0000","last_fullsized":"2024-09-05T13:13:26.209163+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19263488,"num_objects":4703,"num_object_clones":0,"num_object_copies":9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1250,"num_write_kb":5000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4773","reported_seq":4906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.223092+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:26.223092+0000","last_peered":"2024-09-05T13:13:26.223092+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:26.223092+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4773,"log_dups_size":0,"ondisk_log_size":4773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19529728,"num_objects":4768,"num_object_clones":0,"num_object_copies":9536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1228,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":317},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4708","reported_seq":4840,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.265322+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:26.265322+0000","last_peered":"2024-09-05T13:13:26.265322+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:26.265322+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259392,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1027,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":266},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4790","reported_seq":4916,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:26.211113+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:26.211113+0000","last_peered":"2024-09-05T13:13:26.211113+0000","last_clean":"2024-09-05T13:13:26.211113+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:26.211113+0000","last_undegraded":"2024-09-05T13:13:26.211113+0000","last_fullsized":"2024-09-05T13:13:26.211113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4790,"log_dups_size":0,"ondisk_log_size":4790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19611648,"num_objects":4788,"num_object_clones":0,"num_object_copies":9576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":5600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4786","reported_seq":4915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.174963+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:27.174963+0000","last_peered":"2024-09-05T13:13:27.174963+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:27.174963+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4786,"log_dups_size":0,"ondisk_log_size":4786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19574784,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":328}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4741","reported_seq":4872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.268809+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:26.268809+0000","last_peered":"2024-09-05T13:13:26.268809+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:26.268809+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19386368,"num_objects":4733,"num_object_clones":0,"num_object_copies":9466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1108,"num_write_kb":4432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":279},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4743","reported_seq":5186,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.114804+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:27.114804+0000","last_peered":"2024-09-05T13:13:27.114804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:27.114804+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19406848,"num_objects":4738,"num_object_clones":0,"num_object_copies":9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4219},{"shards":"2","objects":517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4717","reported_seq":4834,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.168820+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:27.168820+0000","last_peered":"2024-09-05T13:13:27.168820+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:27.168820+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4717,"log_dups_size":0,"ondisk_log_size":4717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4712","reported_seq":4847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.288164+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:26.288164+0000","last_peered":"2024-09-05T13:13:26.288164+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:26.288164+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292160,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":863,"num_write_kb":3452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":291},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4757","reported_seq":4939,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.237618+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:26.237618+0000","last_peered":"2024-09-05T13:13:26.237618+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:26.237618+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4757,"log_dups_size":0,"ondisk_log_size":4757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19451904,"num_objects":4749,"num_object_clones":0,"num_object_copies":9498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":305},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4844","reported_seq":4977,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.294145+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:26.294145+0000","last_peered":"2024-09-05T13:13:26.294145+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:26.294145+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19824640,"num_objects":4840,"num_object_clones":0,"num_object_copies":9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":974,"num_write_kb":3896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":274},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4855","reported_seq":4988,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:26.257137+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:26.257137+0000","last_peered":"2024-09-05T13:13:26.257137+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:26.257137+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":4855,"log_dups_size":0,"ondisk_log_size":4855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19837411,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1523,"num_write_kb":6092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":279},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":311742947,"num_objects":76110,"num_object_clones":0,"num_object_copies":152220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":76110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31909,"num_write_kb":127636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":628776960,"data_stored":628686635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76191,"ondisk_log_size":76191,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461212,"kb_used_data":294860,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93910628,"statfs":{"total":96636764160,"available":96164483072,"internally_reserved":0,"allocated":301936640,"data_stored":301691550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":170323629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461032,"kb_used_data":298900,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93910808,"statfs":{"total":96636764160,"available":96164667392,"internally_reserved":0,"allocated":306073600,"data_stored":305824433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,1,1,2,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111964,"kb_used_data":17804,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94259876,"statfs":{"total":96636764160,"available":96522113024,"internally_reserved":0,"allocated":18231296,"data_stored":17902643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306216960,"data_stored":306213347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19886080,"data_stored":19802981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302673920,"data_stored":302670307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:31.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4696", 'reported_seq': 4765, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:26.292332+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:13:26.292332+0000', 'last_peered': '2024-09-05T13:13:26.292332+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:13:26.292332+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 4696, 'log_dups_size': 0, 'ondisk_log_size': 4696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19234816, 'num_objects': 4696, 'num_object_clones': 0, 'num_object_copies': 9392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4696, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4696, 'num_write_kb': 18784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 275}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4800", 'reported_seq': 4922, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:27.160722+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:13:27.160722+0000', 'last_peered': '2024-09-05T13:13:27.160722+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:13:27.160722+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 4800, 'log_dups_size': 0, 'ondisk_log_size': 4800, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19644416, 'num_objects': 4796, 'num_object_clones': 0, 'num_object_copies': 9592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2611, 'num_write_kb': 10444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4773", 'reported_seq': 4906, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:26.223092+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:13:26.223092+0000', 'last_peered': '2024-09-05T13:13:26.223092+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:13:26.223092+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 4773, 'log_dups_size': 0, 'ondisk_log_size': 4773, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19529728, 'num_objects': 4768, 'num_object_clones': 0, 'num_object_copies': 9536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1228, 'num_write_kb': 4912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 317}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4708", 'reported_seq': 4840, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:26.265322+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:13:26.265322+0000', 'last_peered': '2024-09-05T13:13:26.265322+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:13:26.265322+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 4708, 'log_dups_size': 0, 'ondisk_log_size': 4708, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19259392, 'num_objects': 4702, 'num_object_clones': 0, 'num_object_copies': 9404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1027, 'num_write_kb': 4108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 266}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:13:31.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4786", 'reported_seq': 4915, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:27.174963+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:13:27.174963+0000', 'last_peered': '2024-09-05T13:13:27.174963+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:13:27.174963+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 4786, 'log_dups_size': 0, 'ondisk_log_size': 4786, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19574784, 'num_objects': 4779, 'num_object_clones': 0, 'num_object_copies': 9558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2440, 'num_write_kb': 9760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 328}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4741", 'reported_seq': 4872, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:26.268809+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:13:26.268809+0000', 'last_peered': '2024-09-05T13:13:26.268809+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:13:26.268809+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 4741, 'log_dups_size': 0, 'ondisk_log_size': 4741, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19386368, 'num_objects': 4733, 'num_object_clones': 0, 'num_object_copies': 9466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1108, 'num_write_kb': 4432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 279}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4743", 'reported_seq': 5186, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:27.114804+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:13:27.114804+0000', 'last_peered': '2024-09-05T13:13:27.114804+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:13:27.114804+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 4743, 'log_dups_size': 0, 'ondisk_log_size': 4743, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19406848, 'num_objects': 4738, 'num_object_clones': 0, 'num_object_copies': 9476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 4219, 'num_objects_unfound': 0, 'num_objects_dirty': 4738, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2338, 'num_write_kb': 9352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 166, 'num_bytes_recovered': 679936, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4219}, {'shards': '2', 'objects': 517}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4717", 'reported_seq': 4834, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:27.168820+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:13:27.168820+0000', 'last_peered': '2024-09-05T13:13:27.168820+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:13:27.168820+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 4717, 'log_dups_size': 0, 'ondisk_log_size': 4717, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19296256, 'num_objects': 4711, 'num_object_clones': 0, 'num_object_copies': 9422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4711, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2226, 'num_write_kb': 8904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 324}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4712", 'reported_seq': 4847, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:26.288164+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:13:26.288164+0000', 'last_peered': '2024-09-05T13:13:26.288164+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:13:26.288164+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 4712, 'log_dups_size': 0, 'ondisk_log_size': 4712, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19292160, 'num_objects': 4710, 'num_object_clones': 0, 'num_object_copies': 9420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 863, 'num_write_kb': 3452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 291}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:31.530 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:13:31.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4757", 'reported_seq': 4939, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:26.237618+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:13:26.237618+0000', 'last_peered': '2024-09-05T13:13:26.237618+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:13:26.237618+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 4757, 'log_dups_size': 0, 'ondisk_log_size': 4757, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19451904, 'num_objects': 4749, 'num_object_clones': 0, 'num_object_copies': 9498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 656, 'num_write_kb': 2624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 305}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:31.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:13:31.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4844", 'reported_seq': 4977, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:26.294145+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:13:26.294145+0000', 'last_peered': '2024-09-05T13:13:26.294145+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:13:26.294145+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 4844, 'log_dups_size': 0, 'ondisk_log_size': 4844, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19824640, 'num_objects': 4840, 'num_object_clones': 0, 'num_object_copies': 9680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 974, 'num_write_kb': 3896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 274}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:31.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:13:31.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4855", 'reported_seq': 4988, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:26.257137+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:13:26.257137+0000', 'last_peered': '2024-09-05T13:13:26.257137+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:13:26.257137+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 4855, 'log_dups_size': 0, 'ondisk_log_size': 4855, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19837411, 'num_objects': 4844, 'num_object_clones': 0, 'num_object_copies': 9688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1523, 'num_write_kb': 6092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 279}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:31.531 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:13:32.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 59 20 79388 79368 5.25401 2.12109 0.0539949 0.0148603 2024-09-05T13:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:32.238 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/155972 objects degraded (31.485%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":139,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77986,"data_bytes":319418870,"bytes_used":1076666368,"bytes_avail":288833626112,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":155972,"degraded_ratio":0.31485138358166853,"misplaced_objects":4219,"misplaced_total":155972,"misplaced_ratio":0.027049726874054318,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":56649,"recovering_keys_per_sec":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"write_bytes_sec":2291902,"read_op_per_sec":0,"write_op_per_sec":559},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:13:32.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:13:32.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:32.802 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:32.802 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:32.828 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-05T13:13:31.526122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":319418870,"num_objects":77986,"num_object_clones":0,"num_object_copies":155972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":77986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33785,"num_write_kb":135134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":692243,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78068,"ondisk_log_size":78068,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1051432,"kb_used_data":620472,"kb_used_omap":12,"kb_used_meta":430899,"kb_avail":282064088,"statfs":{"total":289910292480,"available":288833626112,"internally_reserved":0,"allocated":635363328,"data_stored":634540418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":441240582},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,6,13,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27508736,"num_objects":6716,"num_object_clones":0,"num_object_copies":13432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":6716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6716,"num_write_kb":26864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002577"},"pg_stats":[{"pgid":"2.f","version":"43'4882","reported_seq":4948,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.301024+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:31.301024+0000","last_peered":"2024-09-05T13:13:31.301024+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:31.301024+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4882,"log_dups_size":0,"ondisk_log_size":4882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19996672,"num_objects":4882,"num_object_clones":0,"num_object_copies":9764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4882,"num_write_kb":19528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":461},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4763","reported_seq":4862,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:27.202815+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:27.202815+0000","last_peered":"2024-09-05T13:13:27.202815+0000","last_clean":"2024-09-05T13:13:27.202815+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:27.202815+0000","last_undegraded":"2024-09-05T13:13:27.202815+0000","last_fullsized":"2024-09-05T13:13:27.202815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4763,"log_dups_size":0,"ondisk_log_size":4763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19484672,"num_objects":4757,"num_object_clones":0,"num_object_copies":9514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4800","reported_seq":4922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.160722+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:27.160722+0000","last_peered":"2024-09-05T13:13:27.160722+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:27.160722+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4800,"log_dups_size":0,"ondisk_log_size":4800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2611,"num_write_kb":10444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":304}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4796","reported_seq":4860,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:27.151241+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:27.151241+0000","last_peered":"2024-09-05T13:13:27.151241+0000","last_clean":"2024-09-05T13:13:27.151241+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:27.151241+0000","last_undegraded":"2024-09-05T13:13:27.151241+0000","last_fullsized":"2024-09-05T13:13:27.151241+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4796,"log_dups_size":0,"ondisk_log_size":4796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4796,"num_write_kb":19184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4910","reported_seq":5039,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:31.309156+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:31.309156+0000","last_peered":"2024-09-05T13:13:31.309156+0000","last_clean":"2024-09-05T13:13:31.309156+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:31.309156+0000","last_undegraded":"2024-09-05T13:13:31.309156+0000","last_fullsized":"2024-09-05T13:13:31.309156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4956","reported_seq":5089,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.250792+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:31.250792+0000","last_peered":"2024-09-05T13:13:31.250792+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:31.250792+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20279296,"num_objects":4951,"num_object_clones":0,"num_object_copies":9902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":5644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":500},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4875","reported_seq":5009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.296923+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:31.296923+0000","last_peered":"2024-09-05T13:13:31.296923+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:31.296923+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4875,"log_dups_size":0,"ondisk_log_size":4875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19943424,"num_objects":4869,"num_object_clones":0,"num_object_copies":9738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":433},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4983","reported_seq":5109,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:31.289678+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:31.289678+0000","last_peered":"2024-09-05T13:13:31.289678+0000","last_clean":"2024-09-05T13:13:31.289678+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:31.289678+0000","last_undegraded":"2024-09-05T13:13:31.289678+0000","last_fullsized":"2024-09-05T13:13:31.289678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4983,"log_dups_size":0,"ondisk_log_size":4983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1593,"num_write_kb":6372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4786","reported_seq":4915,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.174963+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:27.174963+0000","last_peered":"2024-09-05T13:13:27.174963+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:27.174963+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4786,"log_dups_size":0,"ondisk_log_size":4786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19574784,"num_objects":4779,"num_object_clones":0,"num_object_copies":9558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":328}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4926","reported_seq":5058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.272950+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:31.272950+0000","last_peered":"2024-09-05T13:13:31.272950+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:31.272950+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20144128,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1293,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4743","reported_seq":5186,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.114804+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:27.114804+0000","last_peered":"2024-09-05T13:13:27.114804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:27.114804+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19406848,"num_objects":4738,"num_object_clones":0,"num_object_copies":9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4219},{"shards":"2","objects":517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4717","reported_seq":4834,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:27.168820+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:27.168820+0000","last_peered":"2024-09-05T13:13:27.168820+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:27.168820+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4717,"log_dups_size":0,"ondisk_log_size":4717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2226,"num_write_kb":8904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4914","reported_seq":5049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.190275+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:31.190275+0000","last_peered":"2024-09-05T13:13:31.190275+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:31.190275+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4914,"log_dups_size":0,"ondisk_log_size":4914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":4912,"num_object_clones":0,"num_object_copies":9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1065,"num_write_kb":4260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":493},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4954","reported_seq":5136,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.329822+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:31.329822+0000","last_peered":"2024-09-05T13:13:31.329822+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:31.329822+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":853,"num_write_kb":3412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":502},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5020","reported_seq":5153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.222943+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:31.222943+0000","last_peered":"2024-09-05T13:13:31.222943+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:31.222943+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20545536,"num_objects":5016,"num_object_clones":0,"num_object_copies":10032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1150,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":450},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5040","reported_seq":5170,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.229762+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:31.229762+0000","last_peered":"2024-09-05T13:13:31.229762+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:31.229762+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5040,"log_dups_size":0,"ondisk_log_size":5040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20595171,"num_objects":5029,"num_object_clones":0,"num_object_copies":10058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1708,"num_write_kb":6832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":319418851,"num_objects":77984,"num_object_clones":0,"num_object_copies":155968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4219,"num_objects_unfound":0,"num_objects_dirty":77984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33783,"num_write_kb":135132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":637902848,"data_stored":637812523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78065,"ondisk_log_size":78065,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461212,"kb_used_data":294860,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93910628,"statfs":{"total":96636764160,"available":96164483072,"internally_reserved":0,"allocated":301936640,"data_stored":301691550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":170323629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478256,"kb_used_data":307808,"kb_used_omap":4,"kb_used_meta":170427,"kb_avail":93893584,"statfs":{"total":96636764160,"available":96147030016,"internally_reserved":0,"allocated":315195392,"data_stored":314946225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":174517930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111964,"kb_used_data":17804,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94259876,"statfs":{"total":96636764160,"available":96522113024,"internally_reserved":0,"allocated":18231296,"data_stored":17902643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315342848,"data_stored":315339235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19886080,"data_stored":19802981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302673920,"data_stored":302670307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:33.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:13:33.193598+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0150021 2024-09-05T13:13:33.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:13:33.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 60 20 79969 79949 5.20427 2.26953 0.0320283 0.0150021 2024-09-05T13:13:34.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 61 20 80590 80570 5.15872 2.42578 0.0138278 0.0151348 2024-09-05T13:13:35.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 62 20 81229 81209 5.11577 2.49609 0.027019 0.0152628 2024-09-05T13:13:35.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:36.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 63 20 81817 81797 5.07103 2.29688 0.0179394 0.0153971 2024-09-05T13:13:36.409 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:36.409 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:36.434 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-05T13:13:35.527000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":323920374,"num_objects":79085,"num_object_clones":0,"num_object_copies":158170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4102,"num_objects_unfound":0,"num_objects_dirty":79085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34884,"num_write_kb":139530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1171475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79167,"ondisk_log_size":79167,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1048832,"kb_used_data":638672,"kb_used_omap":12,"kb_used_meta":410099,"kb_avail":282066688,"statfs":{"total":289910292480,"available":288836288512,"internally_reserved":0,"allocated":654000128,"data_stored":653177218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":419941382},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":24056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002583"},"pg_stats":[{"pgid":"2.f","version":"43'4882","reported_seq":4948,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.301024+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:31.301024+0000","last_peered":"2024-09-05T13:13:31.301024+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:31.301024+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4882,"log_dups_size":0,"ondisk_log_size":4882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19996672,"num_objects":4882,"num_object_clones":0,"num_object_copies":9764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4882,"num_write_kb":19528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":461},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4939","reported_seq":5038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:31.997689+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:31.997689+0000","last_peered":"2024-09-05T13:13:31.997689+0000","last_clean":"2024-09-05T13:13:31.997689+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:31.997689+0000","last_undegraded":"2024-09-05T13:13:31.997689+0000","last_fullsized":"2024-09-05T13:13:31.997689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4939,"log_dups_size":0,"ondisk_log_size":4939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20205568,"num_objects":4933,"num_object_clones":0,"num_object_copies":9866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2949,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4981","reported_seq":5103,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:32.181866+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:32.181866+0000","last_peered":"2024-09-05T13:13:32.181866+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:32.181866+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4981,"log_dups_size":0,"ondisk_log_size":4981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20385792,"num_objects":4977,"num_object_clones":0,"num_object_copies":9954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2792,"num_write_kb":11168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":485}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4977","reported_seq":5041,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:32.194022+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:32.194022+0000","last_peered":"2024-09-05T13:13:32.194022+0000","last_clean":"2024-09-05T13:13:32.194022+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:32.194022+0000","last_undegraded":"2024-09-05T13:13:32.194022+0000","last_fullsized":"2024-09-05T13:13:32.194022+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20385792,"num_objects":4977,"num_object_clones":0,"num_object_copies":9954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4977,"num_write_kb":19908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4910","reported_seq":5039,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:31.309156+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:31.309156+0000","last_peered":"2024-09-05T13:13:31.309156+0000","last_clean":"2024-09-05T13:13:31.309156+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:31.309156+0000","last_undegraded":"2024-09-05T13:13:31.309156+0000","last_fullsized":"2024-09-05T13:13:31.309156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4956","reported_seq":5089,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.250792+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:31.250792+0000","last_peered":"2024-09-05T13:13:31.250792+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:31.250792+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20279296,"num_objects":4951,"num_object_clones":0,"num_object_copies":9902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":5644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":500},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4875","reported_seq":5009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.296923+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:31.296923+0000","last_peered":"2024-09-05T13:13:31.296923+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:31.296923+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4875,"log_dups_size":0,"ondisk_log_size":4875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19943424,"num_objects":4869,"num_object_clones":0,"num_object_copies":9738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":433},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4983","reported_seq":5109,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:31.289678+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:31.289678+0000","last_peered":"2024-09-05T13:13:31.289678+0000","last_clean":"2024-09-05T13:13:31.289678+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:31.289678+0000","last_undegraded":"2024-09-05T13:13:31.289678+0000","last_fullsized":"2024-09-05T13:13:31.289678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4983,"log_dups_size":0,"ondisk_log_size":4983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1593,"num_write_kb":6372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4972","reported_seq":5101,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:32.066695+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:32.066695+0000","last_peered":"2024-09-05T13:13:32.066695+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:32.066695+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20336640,"num_objects":4965,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2626,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4926","reported_seq":5058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.272950+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:31.272950+0000","last_peered":"2024-09-05T13:13:31.272950+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:31.272950+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20144128,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1293,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4929","reported_seq":5606,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:32.162867+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:32.162867+0000","last_peered":"2024-09-05T13:13:32.162867+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:32.162867+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":4102,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2524,"num_write_kb":10096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4102},{"shards":"2","objects":820}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4906","reported_seq":5021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:32.175021+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:32.175021+0000","last_peered":"2024-09-05T13:13:32.175021+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:32.175021+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4906,"log_dups_size":0,"ondisk_log_size":4906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2415,"num_write_kb":9660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":513}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4914","reported_seq":5049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.190275+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:31.190275+0000","last_peered":"2024-09-05T13:13:31.190275+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:31.190275+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4914,"log_dups_size":0,"ondisk_log_size":4914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":4912,"num_object_clones":0,"num_object_copies":9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1065,"num_write_kb":4260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":493},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4954","reported_seq":5136,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.329822+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:31.329822+0000","last_peered":"2024-09-05T13:13:31.329822+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:31.329822+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":853,"num_write_kb":3412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":502},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5020","reported_seq":5153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.222943+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:31.222943+0000","last_peered":"2024-09-05T13:13:31.222943+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:31.222943+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20545536,"num_objects":5016,"num_object_clones":0,"num_object_copies":10032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1150,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":450},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5040","reported_seq":5170,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.229762+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:31.229762+0000","last_peered":"2024-09-05T13:13:31.229762+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:31.229762+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5040,"log_dups_size":0,"ondisk_log_size":5040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20595171,"num_objects":5029,"num_object_clones":0,"num_object_copies":10058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1708,"num_write_kb":6832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":323920355,"num_objects":79083,"num_object_clones":0,"num_object_copies":158166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4102,"num_objects_unfound":0,"num_objects_dirty":79083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34882,"num_write_kb":139528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":653512704,"data_stored":653422379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79164,"ondisk_log_size":79164,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455700,"kb_used_data":301828,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93916140,"statfs":{"total":96636764160,"available":96170127360,"internally_reserved":0,"allocated":309071872,"data_stored":308826782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":157544109},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478256,"kb_used_data":307808,"kb_used_omap":4,"kb_used_meta":170427,"kb_avail":93893584,"statfs":{"total":96636764160,"available":96147030016,"internally_reserved":0,"allocated":315195392,"data_stored":314946225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":174517930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114876,"kb_used_data":29036,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94256964,"statfs":{"total":96636764160,"available":96519131136,"internally_reserved":0,"allocated":29732864,"data_stored":29404211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":87879343},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315342848,"data_stored":315339235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29405184,"data_stored":29322085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308764672,"data_stored":308761059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:36.435 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:37.001 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:37.004 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:37.032 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-05T13:13:35.527000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":323920374,"num_objects":79085,"num_object_clones":0,"num_object_copies":158170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4102,"num_objects_unfound":0,"num_objects_dirty":79085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34884,"num_write_kb":139530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1171475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79167,"ondisk_log_size":79167,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1048832,"kb_used_data":638672,"kb_used_omap":12,"kb_used_meta":410099,"kb_avail":282066688,"statfs":{"total":289910292480,"available":288836288512,"internally_reserved":0,"allocated":654000128,"data_stored":653177218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":419941382},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":24056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002583"},"pg_stats":[{"pgid":"2.f","version":"43'4882","reported_seq":4948,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.301024+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:31.301024+0000","last_peered":"2024-09-05T13:13:31.301024+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:31.301024+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4882,"log_dups_size":0,"ondisk_log_size":4882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19996672,"num_objects":4882,"num_object_clones":0,"num_object_copies":9764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4882,"num_write_kb":19528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":461},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4939","reported_seq":5038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:31.997689+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:31.997689+0000","last_peered":"2024-09-05T13:13:31.997689+0000","last_clean":"2024-09-05T13:13:31.997689+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:31.997689+0000","last_undegraded":"2024-09-05T13:13:31.997689+0000","last_fullsized":"2024-09-05T13:13:31.997689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":4939,"log_dups_size":0,"ondisk_log_size":4939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20205568,"num_objects":4933,"num_object_clones":0,"num_object_copies":9866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2949,"num_write_kb":11796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4981","reported_seq":5103,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:32.181866+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:32.181866+0000","last_peered":"2024-09-05T13:13:32.181866+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:32.181866+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":4981,"log_dups_size":0,"ondisk_log_size":4981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20385792,"num_objects":4977,"num_object_clones":0,"num_object_copies":9954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2792,"num_write_kb":11168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":485}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4977","reported_seq":5041,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:32.194022+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:32.194022+0000","last_peered":"2024-09-05T13:13:32.194022+0000","last_clean":"2024-09-05T13:13:32.194022+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:32.194022+0000","last_undegraded":"2024-09-05T13:13:32.194022+0000","last_fullsized":"2024-09-05T13:13:32.194022+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20385792,"num_objects":4977,"num_object_clones":0,"num_object_copies":9954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4977,"num_write_kb":19908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4910","reported_seq":5039,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:31.309156+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:31.309156+0000","last_peered":"2024-09-05T13:13:31.309156+0000","last_clean":"2024-09-05T13:13:31.309156+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:31.309156+0000","last_undegraded":"2024-09-05T13:13:31.309156+0000","last_fullsized":"2024-09-05T13:13:31.309156+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4956","reported_seq":5089,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.250792+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:31.250792+0000","last_peered":"2024-09-05T13:13:31.250792+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:31.250792+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20279296,"num_objects":4951,"num_object_clones":0,"num_object_copies":9902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":5644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":500},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4875","reported_seq":5009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.296923+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:31.296923+0000","last_peered":"2024-09-05T13:13:31.296923+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:31.296923+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":4875,"log_dups_size":0,"ondisk_log_size":4875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19943424,"num_objects":4869,"num_object_clones":0,"num_object_copies":9738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":433},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4983","reported_seq":5109,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:31.289678+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:31.289678+0000","last_peered":"2024-09-05T13:13:31.289678+0000","last_clean":"2024-09-05T13:13:31.289678+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:31.289678+0000","last_undegraded":"2024-09-05T13:13:31.289678+0000","last_fullsized":"2024-09-05T13:13:31.289678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":4983,"log_dups_size":0,"ondisk_log_size":4983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1593,"num_write_kb":6372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4972","reported_seq":5101,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:32.066695+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:32.066695+0000","last_peered":"2024-09-05T13:13:32.066695+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:32.066695+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20336640,"num_objects":4965,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2626,"num_write_kb":10504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4926","reported_seq":5058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.272950+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:31.272950+0000","last_peered":"2024-09-05T13:13:31.272950+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:31.272950+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20144128,"num_objects":4918,"num_object_clones":0,"num_object_copies":9836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1293,"num_write_kb":5172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4929","reported_seq":5606,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:32.162867+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:32.162867+0000","last_peered":"2024-09-05T13:13:32.162867+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:32.162867+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20168704,"num_objects":4924,"num_object_clones":0,"num_object_copies":9848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":4102,"num_objects_unfound":0,"num_objects_dirty":4924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2524,"num_write_kb":10096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4102},{"shards":"2","objects":820}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4906","reported_seq":5021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:32.175021+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:32.175021+0000","last_peered":"2024-09-05T13:13:32.175021+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:32.175021+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":4906,"log_dups_size":0,"ondisk_log_size":4906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2415,"num_write_kb":9660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":513}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4914","reported_seq":5049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.190275+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:31.190275+0000","last_peered":"2024-09-05T13:13:31.190275+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:31.190275+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":4914,"log_dups_size":0,"ondisk_log_size":4914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":4912,"num_object_clones":0,"num_object_copies":9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1065,"num_write_kb":4260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":493},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4954","reported_seq":5136,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.329822+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:31.329822+0000","last_peered":"2024-09-05T13:13:31.329822+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:31.329822+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":853,"num_write_kb":3412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":502},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5020","reported_seq":5153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.222943+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:31.222943+0000","last_peered":"2024-09-05T13:13:31.222943+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:31.222943+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20545536,"num_objects":5016,"num_object_clones":0,"num_object_copies":10032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1150,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":450},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5040","reported_seq":5170,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:31.229762+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:31.229762+0000","last_peered":"2024-09-05T13:13:31.229762+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:31.229762+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5040,"log_dups_size":0,"ondisk_log_size":5040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20595171,"num_objects":5029,"num_object_clones":0,"num_object_copies":10058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1708,"num_write_kb":6832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":464},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":323920355,"num_objects":79083,"num_object_clones":0,"num_object_copies":158166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":4102,"num_objects_unfound":0,"num_objects_dirty":79083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34882,"num_write_kb":139528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":653512704,"data_stored":653422379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79164,"ondisk_log_size":79164,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455700,"kb_used_data":301828,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93916140,"statfs":{"total":96636764160,"available":96170127360,"internally_reserved":0,"allocated":309071872,"data_stored":308826782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":157544109},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478256,"kb_used_data":307808,"kb_used_omap":4,"kb_used_meta":170427,"kb_avail":93893584,"statfs":{"total":96636764160,"available":96147030016,"internally_reserved":0,"allocated":315195392,"data_stored":314946225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":174517930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114876,"kb_used_data":29036,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94256964,"statfs":{"total":96636764160,"available":96519131136,"internally_reserved":0,"allocated":29732864,"data_stored":29404211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":87879343},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315342848,"data_stored":315339235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29405184,"data_stored":29322085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308764672,"data_stored":308761059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:37.033 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:13:37.033 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4882", 'reported_seq': 4948, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:31.301024+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:13:31.301024+0000', 'last_peered': '2024-09-05T13:13:31.301024+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:13:31.301024+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 4882, 'log_dups_size': 0, 'ondisk_log_size': 4882, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19996672, 'num_objects': 4882, 'num_object_clones': 0, 'num_object_copies': 9764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4882, 'num_write_kb': 19528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 461}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:37.033 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:13:37.033 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4981", 'reported_seq': 5103, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:32.181866+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:13:32.181866+0000', 'last_peered': '2024-09-05T13:13:32.181866+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:13:32.181866+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 4981, 'log_dups_size': 0, 'ondisk_log_size': 4981, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20385792, 'num_objects': 4977, 'num_object_clones': 0, 'num_object_copies': 9954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2792, 'num_write_kb': 11168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 485}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:37.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:13:37.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4956", 'reported_seq': 5089, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:31.250792+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:13:31.250792+0000', 'last_peered': '2024-09-05T13:13:31.250792+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:13:31.250792+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 4956, 'log_dups_size': 0, 'ondisk_log_size': 4956, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20279296, 'num_objects': 4951, 'num_object_clones': 0, 'num_object_copies': 9902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1411, 'num_write_kb': 5644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 500}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:37.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:13:37.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4875", 'reported_seq': 5009, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:31.296923+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:13:31.296923+0000', 'last_peered': '2024-09-05T13:13:31.296923+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:13:31.296923+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 4875, 'log_dups_size': 0, 'ondisk_log_size': 4875, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19943424, 'num_objects': 4869, 'num_object_clones': 0, 'num_object_copies': 9738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1194, 'num_write_kb': 4776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 433}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:37.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:13:37.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4972", 'reported_seq': 5101, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:32.066695+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:13:32.066695+0000', 'last_peered': '2024-09-05T13:13:32.066695+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:13:32.066695+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 4972, 'log_dups_size': 0, 'ondisk_log_size': 4972, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20336640, 'num_objects': 4965, 'num_object_clones': 0, 'num_object_copies': 9930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4965, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2626, 'num_write_kb': 10504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4926", 'reported_seq': 5058, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:31.272950+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:13:31.272950+0000', 'last_peered': '2024-09-05T13:13:31.272950+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:13:31.272950+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 4926, 'log_dups_size': 0, 'ondisk_log_size': 4926, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20144128, 'num_objects': 4918, 'num_object_clones': 0, 'num_object_copies': 9836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1293, 'num_write_kb': 5172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 464}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4929", 'reported_seq': 5606, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:32.162867+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:13:32.162867+0000', 'last_peered': '2024-09-05T13:13:32.162867+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:13:32.162867+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 4929, 'log_dups_size': 0, 'ondisk_log_size': 4929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20168704, 'num_objects': 4924, 'num_object_clones': 0, 'num_object_copies': 9848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 4102, 'num_objects_unfound': 0, 'num_objects_dirty': 4924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2524, 'num_write_kb': 10096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 283, 'num_bytes_recovered': 1159168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4102}, {'shards': '2', 'objects': 820}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4906", 'reported_seq': 5021, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:32.175021+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:13:32.175021+0000', 'last_peered': '2024-09-05T13:13:32.175021+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:13:32.175021+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 4906, 'log_dups_size': 0, 'ondisk_log_size': 4906, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20070400, 'num_objects': 4900, 'num_object_clones': 0, 'num_object_copies': 9800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2415, 'num_write_kb': 9660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 513}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4914", 'reported_seq': 5049, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:31.190275+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:13:31.190275+0000', 'last_peered': '2024-09-05T13:13:31.190275+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:13:31.190275+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 4914, 'log_dups_size': 0, 'ondisk_log_size': 4914, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20119552, 'num_objects': 4912, 'num_object_clones': 0, 'num_object_copies': 9824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1065, 'num_write_kb': 4260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 493}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:13:37.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4954", 'reported_seq': 5136, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:31.329822+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:13:31.329822+0000', 'last_peered': '2024-09-05T13:13:31.329822+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:13:31.329822+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 4954, 'log_dups_size': 0, 'ondisk_log_size': 4954, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20258816, 'num_objects': 4946, 'num_object_clones': 0, 'num_object_copies': 9892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 853, 'num_write_kb': 3412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 502}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:37.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:13:37.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5020", 'reported_seq': 5153, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:31.222943+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:13:31.222943+0000', 'last_peered': '2024-09-05T13:13:31.222943+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:13:31.222943+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 5020, 'log_dups_size': 0, 'ondisk_log_size': 5020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20545536, 'num_objects': 5016, 'num_object_clones': 0, 'num_object_copies': 10032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1150, 'num_write_kb': 4600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 450}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:37.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:13:37.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5040", 'reported_seq': 5170, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:31.229762+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:13:31.229762+0000', 'last_peered': '2024-09-05T13:13:31.229762+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:13:31.229762+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 5040, 'log_dups_size': 0, 'ondisk_log_size': 5040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20595171, 'num_objects': 5029, 'num_object_clones': 0, 'num_object_copies': 10058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1708, 'num_write_kb': 6832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 464}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:37.036 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:13:37.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 64 20 82351 82331 5.02439 2.08594 0.0131365 0.0155363 2024-09-05T13:13:37.717 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:37.717 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/158170 objects degraded (31.048%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":144,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":79085,"data_bytes":323920374,"bytes_used":1074003968,"bytes_avail":288836288512,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":158170,"degraded_ratio":0.3104760700512107,"misplaced_objects":4102,"misplaced_total":158170,"misplaced_ratio":0.025934121514825822,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":74735,"recovering_keys_per_sec":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"write_bytes_sec":2052336,"read_op_per_sec":0,"write_op_per_sec":501},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:13:37.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:13:37.741 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:38.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 65 20 82892 82872 4.97946 2.11328 0.0178486 0.0156814 2024-09-05T13:13:38.303 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:38.304 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:38.326 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-05T13:13:37.527417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336101878,"num_objects":82059,"num_object_clones":0,"num_object_copies":164118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":82059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37858,"num_write_kb":151426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":1617939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82141,"ondisk_log_size":82141,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1073296,"kb_used_data":653408,"kb_used_omap":12,"kb_used_meta":419827,"kb_avail":282042224,"statfs":{"total":289910292480,"available":288811237376,"internally_reserved":0,"allocated":669089792,"data_stored":668266882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":429902854},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,7,2,1,4,4,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":36814848,"num_objects":8988,"num_object_clones":0,"num_object_copies":17976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-328,"num_objects_unfound":0,"num_objects_dirty":8988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8988,"num_write_kb":35952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1343488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002589"},"pg_stats":[{"pgid":"2.f","version":"43'5072","reported_seq":5140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:36.322660+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:36.322660+0000","last_peered":"2024-09-05T13:13:36.322660+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:36.322660+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5072,"log_dups_size":0,"ondisk_log_size":5072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":651},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5120","reported_seq":5219,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:37.114972+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:37.114972+0000","last_peered":"2024-09-05T13:13:37.114972+0000","last_clean":"2024-09-05T13:13:37.114972+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:37.114972+0000","last_undegraded":"2024-09-05T13:13:37.114972+0000","last_fullsized":"2024-09-05T13:13:37.114972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3130,"num_write_kb":12520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5163","reported_seq":5285,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.209618+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:37.209618+0000","last_peered":"2024-09-05T13:13:37.209618+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:37.209618+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5163,"log_dups_size":0,"ondisk_log_size":5163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131264,"num_objects":5159,"num_object_clones":0,"num_object_copies":10318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5154","reported_seq":5218,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:37.110566+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:37.110566+0000","last_peered":"2024-09-05T13:13:37.110566+0000","last_clean":"2024-09-05T13:13:37.110566+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:37.110566+0000","last_undegraded":"2024-09-05T13:13:37.110566+0000","last_fullsized":"2024-09-05T13:13:37.110566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5154,"log_dups_size":0,"ondisk_log_size":5154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21110784,"num_objects":5154,"num_object_clones":0,"num_object_copies":10308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5154,"num_write_kb":20616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5128","reported_seq":5257,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:36.290221+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:36.290221+0000","last_peered":"2024-09-05T13:13:36.290221+0000","last_clean":"2024-09-05T13:13:36.290221+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:36.290221+0000","last_undegraded":"2024-09-05T13:13:36.290221+0000","last_fullsized":"2024-09-05T13:13:36.290221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1668,"num_write_kb":6672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5143","reported_seq":5276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:36.215074+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:36.215074+0000","last_peered":"2024-09-05T13:13:36.215074+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:36.215074+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5143,"log_dups_size":0,"ondisk_log_size":5143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21045248,"num_objects":5138,"num_object_clones":0,"num_object_copies":10276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1598,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":687},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5062","reported_seq":5193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:36.350120+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:36.350120+0000","last_peered":"2024-09-05T13:13:36.350120+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:36.350120+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5062,"log_dups_size":0,"ondisk_log_size":5062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20709376,"num_objects":5056,"num_object_clones":0,"num_object_copies":10112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1381,"num_write_kb":5524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":620},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5180","reported_seq":5306,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:36.337327+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:36.337327+0000","last_peered":"2024-09-05T13:13:36.337327+0000","last_clean":"2024-09-05T13:13:36.337327+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:36.337327+0000","last_undegraded":"2024-09-05T13:13:36.337327+0000","last_fullsized":"2024-09-05T13:13:36.337327+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5180,"log_dups_size":0,"ondisk_log_size":5180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21209088,"num_objects":5178,"num_object_clones":0,"num_object_copies":10356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1790,"num_write_kb":7160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5153","reported_seq":5282,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.185238+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:37.185238+0000","last_peered":"2024-09-05T13:13:37.185238+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:37.185238+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078016,"num_objects":5146,"num_object_clones":0,"num_object_copies":10292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2807,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5126","reported_seq":5257,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:36.277077+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:36.277077+0000","last_peered":"2024-09-05T13:13:36.277077+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:36.277077+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5126,"log_dups_size":0,"ondisk_log_size":5126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1493,"num_write_kb":5972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":664},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5117","reported_seq":6012,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.216038+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:37.216038+0000","last_peered":"2024-09-05T13:13:37.216038+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:37.216038+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5117,"log_dups_size":0,"ondisk_log_size":5117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2712,"num_write_kb":10848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3993},{"shards":"2","objects":1117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5095","reported_seq":5212,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.209676+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:37.209676+0000","last_peered":"2024-09-05T13:13:37.209676+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:37.209676+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5095,"log_dups_size":0,"ondisk_log_size":5095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20844544,"num_objects":5089,"num_object_clones":0,"num_object_copies":10178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5091","reported_seq":5226,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:36.319971+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:36.319971+0000","last_peered":"2024-09-05T13:13:36.319971+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:36.319971+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20844544,"num_objects":5089,"num_object_clones":0,"num_object_copies":10178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1242,"num_write_kb":4968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":670},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5136","reported_seq":5318,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:36.349094+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:36.349094+0000","last_peered":"2024-09-05T13:13:36.349094+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:36.349094+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5136,"log_dups_size":0,"ondisk_log_size":5136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21004288,"num_objects":5128,"num_object_clones":0,"num_object_copies":10256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":684},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5189","reported_seq":5322,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:36.236960+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:36.236960+0000","last_peered":"2024-09-05T13:13:36.236960+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:36.236960+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5189,"log_dups_size":0,"ondisk_log_size":5189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21237760,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1319,"num_write_kb":5276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":619},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5209","reported_seq":5343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:36.343556+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:36.343556+0000","last_peered":"2024-09-05T13:13:36.343556+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:36.343556+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5209,"log_dups_size":0,"ondisk_log_size":5209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21287395,"num_objects":5198,"num_object_clones":0,"num_object_copies":10396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1877,"num_write_kb":7508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":633},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":336101859,"num_objects":82057,"num_object_clones":0,"num_object_copies":164114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":82057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37856,"num_write_kb":151424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":1617920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":668844032,"data_stored":668753707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82138,"ondisk_log_size":82138,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465848,"kb_used_data":307816,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93905992,"statfs":{"total":96636764160,"available":96159735808,"internally_reserved":0,"allocated":315203584,"data_stored":314958494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":161803949},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":316556,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":324153344,"data_stored":323904177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,7,0,0,3,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114876,"kb_used_data":29036,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94256964,"statfs":{"total":96636764160,"available":96519131136,"internally_reserved":0,"allocated":29732864,"data_stored":29404211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":87879343},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":324493312,"data_stored":324489699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29405184,"data_stored":29322085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314945536,"data_stored":314941923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:39.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 66 20 83495 83475 4.9397 2.35547 0.0162419 0.0158081 2024-09-05T13:13:40.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 67 20 84107 84087 4.90145 2.39062 0.0186746 0.0159311 2024-09-05T13:13:41.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 68 20 84709 84689 4.86395 2.35156 0.0310165 0.0160541 2024-09-05T13:13:41.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:41.869 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:41.893 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-05T13:13:41.528642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343523830,"num_objects":83871,"num_object_clones":0,"num_object_copies":167742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":83871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39670,"num_write_kb":158674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":1617939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83953,"ondisk_log_size":83953,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1103148,"kb_used_data":670780,"kb_used_omap":12,"kb_used_meta":432307,"kb_avail":282012372,"statfs":{"total":289910292480,"available":288780668928,"internally_reserved":0,"allocated":686878720,"data_stored":686055810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":442682374},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,1,6,7,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-226,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7759,"num_write_kb":31036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":925696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002964"},"pg_stats":[{"pgid":"2.f","version":"43'5240","reported_seq":5309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.216755+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:41.216755+0000","last_peered":"2024-09-05T13:13:41.216755+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:41.216755+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21463040,"num_objects":5240,"num_object_clones":0,"num_object_copies":10480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5240,"num_write_kb":20960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":819},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5120","reported_seq":5219,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:37.114972+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:37.114972+0000","last_peered":"2024-09-05T13:13:37.114972+0000","last_clean":"2024-09-05T13:13:37.114972+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:37.114972+0000","last_undegraded":"2024-09-05T13:13:37.114972+0000","last_fullsized":"2024-09-05T13:13:37.114972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3130,"num_write_kb":12520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5163","reported_seq":5285,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.209618+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:37.209618+0000","last_peered":"2024-09-05T13:13:37.209618+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:37.209618+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5163,"log_dups_size":0,"ondisk_log_size":5163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131264,"num_objects":5159,"num_object_clones":0,"num_object_copies":10318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5154","reported_seq":5218,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:37.110566+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:37.110566+0000","last_peered":"2024-09-05T13:13:37.110566+0000","last_clean":"2024-09-05T13:13:37.110566+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:37.110566+0000","last_undegraded":"2024-09-05T13:13:37.110566+0000","last_fullsized":"2024-09-05T13:13:37.110566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5154,"log_dups_size":0,"ondisk_log_size":5154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21110784,"num_objects":5154,"num_object_clones":0,"num_object_copies":10308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5154,"num_write_kb":20616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5310","reported_seq":5439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:41.340295+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:41.340295+0000","last_peered":"2024-09-05T13:13:41.340295+0000","last_clean":"2024-09-05T13:13:41.340295+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:41.340295+0000","last_undegraded":"2024-09-05T13:13:41.340295+0000","last_fullsized":"2024-09-05T13:13:41.340295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21721088,"num_objects":5303,"num_object_clones":0,"num_object_copies":10606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":7400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5318","reported_seq":5451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.290704+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:41.290704+0000","last_peered":"2024-09-05T13:13:41.290704+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:41.290704+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5318,"log_dups_size":0,"ondisk_log_size":5318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":862},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5221","reported_seq":5355,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.370820+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:41.370820+0000","last_peered":"2024-09-05T13:13:41.370820+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:41.370820+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5221,"log_dups_size":0,"ondisk_log_size":5221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21360640,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1540,"num_write_kb":6160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":779},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5367","reported_seq":5493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:41.348839+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:41.348839+0000","last_peered":"2024-09-05T13:13:41.348839+0000","last_clean":"2024-09-05T13:13:41.348839+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:41.348839+0000","last_undegraded":"2024-09-05T13:13:41.348839+0000","last_fullsized":"2024-09-05T13:13:41.348839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5367,"log_dups_size":0,"ondisk_log_size":5367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21975040,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":7908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5153","reported_seq":5282,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.185238+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:37.185238+0000","last_peered":"2024-09-05T13:13:37.185238+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:37.185238+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078016,"num_objects":5146,"num_object_clones":0,"num_object_copies":10292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2807,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5302","reported_seq":5434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.316355+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:41.316355+0000","last_peered":"2024-09-05T13:13:41.316355+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:41.316355+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684224,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":6676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5117","reported_seq":6012,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.216038+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:37.216038+0000","last_peered":"2024-09-05T13:13:37.216038+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:37.216038+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5117,"log_dups_size":0,"ondisk_log_size":5117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2712,"num_write_kb":10848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3993},{"shards":"2","objects":1117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5095","reported_seq":5212,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.209676+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:37.209676+0000","last_peered":"2024-09-05T13:13:37.209676+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:37.209676+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5095,"log_dups_size":0,"ondisk_log_size":5095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20844544,"num_objects":5089,"num_object_clones":0,"num_object_copies":10178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5277","reported_seq":5412,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.351239+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:41.351239+0000","last_peered":"2024-09-05T13:13:41.351239+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:41.351239+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5277,"log_dups_size":0,"ondisk_log_size":5277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1428,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5332","reported_seq":5514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.327431+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:41.327431+0000","last_peered":"2024-09-05T13:13:41.327431+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:41.327431+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5332,"log_dups_size":0,"ondisk_log_size":5332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":880},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5376","reported_seq":5509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.307397+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:41.307397+0000","last_peered":"2024-09-05T13:13:41.307397+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:41.307397+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5376,"log_dups_size":0,"ondisk_log_size":5376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":806},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5405","reported_seq":5539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.365443+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:41.365443+0000","last_peered":"2024-09-05T13:13:41.365443+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:41.365443+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22090211,"num_objects":5394,"num_object_clones":0,"num_object_copies":10788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2073,"num_write_kb":8292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":829},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":343523811,"num_objects":83869,"num_object_clones":0,"num_object_copies":167738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":83869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39668,"num_write_kb":158672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":1617920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":686989312,"data_stored":686898987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83950,"ondisk_log_size":83950,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465848,"kb_used_data":307816,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93905992,"statfs":{"total":96636764160,"available":96159735808,"internally_reserved":0,"allocated":315203584,"data_stored":314958494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":161803949},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505172,"kb_used_data":324996,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93866668,"statfs":{"total":96636764160,"available":96119468032,"internally_reserved":0,"allocated":332795904,"data_stored":332546737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132128,"kb_used_data":37968,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94239712,"statfs":{"total":96636764160,"available":96501465088,"internally_reserved":0,"allocated":38879232,"data_stored":38550579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":333352960,"data_stored":333349347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38690816,"data_stored":38607717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314945536,"data_stored":314941923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:41.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:42.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 69 20 85249 85229 4.82403 2.10938 0.0393239 0.0161857 2024-09-05T13:13:42.457 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:42.480 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-05T13:13:41.528642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343523830,"num_objects":83871,"num_object_clones":0,"num_object_copies":167742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":83871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39670,"num_write_kb":158674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":1617939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83953,"ondisk_log_size":83953,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1103148,"kb_used_data":670780,"kb_used_omap":12,"kb_used_meta":432307,"kb_avail":282012372,"statfs":{"total":289910292480,"available":288780668928,"internally_reserved":0,"allocated":686878720,"data_stored":686055810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":442682374},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,1,6,7,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-226,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7759,"num_write_kb":31036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":925696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002964"},"pg_stats":[{"pgid":"2.f","version":"43'5240","reported_seq":5309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.216755+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:41.216755+0000","last_peered":"2024-09-05T13:13:41.216755+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:41.216755+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21463040,"num_objects":5240,"num_object_clones":0,"num_object_copies":10480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5240,"num_write_kb":20960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":819},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5120","reported_seq":5219,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:37.114972+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:37.114972+0000","last_peered":"2024-09-05T13:13:37.114972+0000","last_clean":"2024-09-05T13:13:37.114972+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:37.114972+0000","last_undegraded":"2024-09-05T13:13:37.114972+0000","last_fullsized":"2024-09-05T13:13:37.114972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3130,"num_write_kb":12520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5163","reported_seq":5285,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.209618+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:37.209618+0000","last_peered":"2024-09-05T13:13:37.209618+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:37.209618+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5163,"log_dups_size":0,"ondisk_log_size":5163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131264,"num_objects":5159,"num_object_clones":0,"num_object_copies":10318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5154","reported_seq":5218,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:37.110566+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:37.110566+0000","last_peered":"2024-09-05T13:13:37.110566+0000","last_clean":"2024-09-05T13:13:37.110566+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:37.110566+0000","last_undegraded":"2024-09-05T13:13:37.110566+0000","last_fullsized":"2024-09-05T13:13:37.110566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5154,"log_dups_size":0,"ondisk_log_size":5154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21110784,"num_objects":5154,"num_object_clones":0,"num_object_copies":10308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5154,"num_write_kb":20616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5310","reported_seq":5439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:41.340295+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:41.340295+0000","last_peered":"2024-09-05T13:13:41.340295+0000","last_clean":"2024-09-05T13:13:41.340295+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:41.340295+0000","last_undegraded":"2024-09-05T13:13:41.340295+0000","last_fullsized":"2024-09-05T13:13:41.340295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21721088,"num_objects":5303,"num_object_clones":0,"num_object_copies":10606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":7400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5318","reported_seq":5451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.290704+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:41.290704+0000","last_peered":"2024-09-05T13:13:41.290704+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:41.290704+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5318,"log_dups_size":0,"ondisk_log_size":5318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":862},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5221","reported_seq":5355,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.370820+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:41.370820+0000","last_peered":"2024-09-05T13:13:41.370820+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:41.370820+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5221,"log_dups_size":0,"ondisk_log_size":5221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21360640,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1540,"num_write_kb":6160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":779},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5367","reported_seq":5493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:41.348839+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:41.348839+0000","last_peered":"2024-09-05T13:13:41.348839+0000","last_clean":"2024-09-05T13:13:41.348839+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:41.348839+0000","last_undegraded":"2024-09-05T13:13:41.348839+0000","last_fullsized":"2024-09-05T13:13:41.348839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5367,"log_dups_size":0,"ondisk_log_size":5367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21975040,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":7908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5153","reported_seq":5282,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.185238+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:37.185238+0000","last_peered":"2024-09-05T13:13:37.185238+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:37.185238+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078016,"num_objects":5146,"num_object_clones":0,"num_object_copies":10292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2807,"num_write_kb":11228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5302","reported_seq":5434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.316355+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:41.316355+0000","last_peered":"2024-09-05T13:13:41.316355+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:41.316355+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684224,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":6676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5117","reported_seq":6012,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.216038+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:37.216038+0000","last_peered":"2024-09-05T13:13:37.216038+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:37.216038+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5117,"log_dups_size":0,"ondisk_log_size":5117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2712,"num_write_kb":10848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3993},{"shards":"2","objects":1117}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5095","reported_seq":5212,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:37.209676+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:37.209676+0000","last_peered":"2024-09-05T13:13:37.209676+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:37.209676+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5095,"log_dups_size":0,"ondisk_log_size":5095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20844544,"num_objects":5089,"num_object_clones":0,"num_object_copies":10178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":10416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5277","reported_seq":5412,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.351239+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:41.351239+0000","last_peered":"2024-09-05T13:13:41.351239+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:41.351239+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5277,"log_dups_size":0,"ondisk_log_size":5277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1428,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5332","reported_seq":5514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.327431+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:41.327431+0000","last_peered":"2024-09-05T13:13:41.327431+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:41.327431+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5332,"log_dups_size":0,"ondisk_log_size":5332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":880},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5376","reported_seq":5509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.307397+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:41.307397+0000","last_peered":"2024-09-05T13:13:41.307397+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:41.307397+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5376,"log_dups_size":0,"ondisk_log_size":5376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":806},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5405","reported_seq":5539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.365443+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:41.365443+0000","last_peered":"2024-09-05T13:13:41.365443+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:41.365443+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22090211,"num_objects":5394,"num_object_clones":0,"num_object_copies":10788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2073,"num_write_kb":8292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":829},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":343523811,"num_objects":83869,"num_object_clones":0,"num_object_copies":167738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3993,"num_objects_unfound":0,"num_objects_dirty":83869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39668,"num_write_kb":158672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":1617920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":686989312,"data_stored":686898987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83950,"ondisk_log_size":83950,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465848,"kb_used_data":307816,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93905992,"statfs":{"total":96636764160,"available":96159735808,"internally_reserved":0,"allocated":315203584,"data_stored":314958494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":161803949},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505172,"kb_used_data":324996,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93866668,"statfs":{"total":96636764160,"available":96119468032,"internally_reserved":0,"allocated":332795904,"data_stored":332546737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132128,"kb_used_data":37968,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94239712,"statfs":{"total":96636764160,"available":96501465088,"internally_reserved":0,"allocated":38879232,"data_stored":38550579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":333352960,"data_stored":333349347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38690816,"data_stored":38607717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314945536,"data_stored":314941923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:42.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5240", 'reported_seq': 5309, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:41.216755+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:13:41.216755+0000', 'last_peered': '2024-09-05T13:13:41.216755+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:13:41.216755+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 5240, 'log_dups_size': 0, 'ondisk_log_size': 5240, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21463040, 'num_objects': 5240, 'num_object_clones': 0, 'num_object_copies': 10480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5240, 'num_write_kb': 20960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 819}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5163", 'reported_seq': 5285, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:37.209618+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:13:37.209618+0000', 'last_peered': '2024-09-05T13:13:37.209618+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:13:37.209618+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 5163, 'log_dups_size': 0, 'ondisk_log_size': 5163, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21131264, 'num_objects': 5159, 'num_object_clones': 0, 'num_object_copies': 10318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2974, 'num_write_kb': 11896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 667}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5318", 'reported_seq': 5451, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:41.290704+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:13:41.290704+0000', 'last_peered': '2024-09-05T13:13:41.290704+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:13:41.290704+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 5318, 'log_dups_size': 0, 'ondisk_log_size': 5318, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21762048, 'num_objects': 5313, 'num_object_clones': 0, 'num_object_copies': 10626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1773, 'num_write_kb': 7092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 862}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5221", 'reported_seq': 5355, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:41.370820+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:13:41.370820+0000', 'last_peered': '2024-09-05T13:13:41.370820+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:13:41.370820+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 5221, 'log_dups_size': 0, 'ondisk_log_size': 5221, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21360640, 'num_objects': 5215, 'num_object_clones': 0, 'num_object_copies': 10430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5215, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1540, 'num_write_kb': 6160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 779}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:13:42.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5153", 'reported_seq': 5282, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:37.185238+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:13:37.185238+0000', 'last_peered': '2024-09-05T13:13:37.185238+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:13:37.185238+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 5153, 'log_dups_size': 0, 'ondisk_log_size': 5153, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21078016, 'num_objects': 5146, 'num_object_clones': 0, 'num_object_copies': 10292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2807, 'num_write_kb': 11228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 695}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:42.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:13:42.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5302", 'reported_seq': 5434, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:41.316355+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:13:41.316355+0000', 'last_peered': '2024-09-05T13:13:41.316355+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:13:41.316355+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 5302, 'log_dups_size': 0, 'ondisk_log_size': 5302, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21684224, 'num_objects': 5294, 'num_object_clones': 0, 'num_object_copies': 10588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1669, 'num_write_kb': 6676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 840}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:42.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:13:42.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5117", 'reported_seq': 6012, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:37.216038+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:13:37.216038+0000', 'last_peered': '2024-09-05T13:13:37.216038+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:13:37.216038+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 5117, 'log_dups_size': 0, 'ondisk_log_size': 5117, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20938752, 'num_objects': 5112, 'num_object_clones': 0, 'num_object_copies': 10224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3993, 'num_objects_unfound': 0, 'num_objects_dirty': 5112, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2712, 'num_write_kb': 10848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 392, 'num_bytes_recovered': 1605632, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3993}, {'shards': '2', 'objects': 1117}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:42.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:13:42.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5095", 'reported_seq': 5212, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:37.209676+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:13:37.209676+0000', 'last_peered': '2024-09-05T13:13:37.209676+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:13:37.209676+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 5095, 'log_dups_size': 0, 'ondisk_log_size': 5095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20844544, 'num_objects': 5089, 'num_object_clones': 0, 'num_object_copies': 10178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2604, 'num_write_kb': 10416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 702}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:42.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:13:42.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5277", 'reported_seq': 5412, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:41.351239+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:13:41.351239+0000', 'last_peered': '2024-09-05T13:13:41.351239+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:13:41.351239+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 5277, 'log_dups_size': 0, 'ondisk_log_size': 5277, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21606400, 'num_objects': 5275, 'num_object_clones': 0, 'num_object_copies': 10550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5275, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1428, 'num_write_kb': 5712, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 856}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:42.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:13:42.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5332", 'reported_seq': 5514, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:41.327431+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:13:41.327431+0000', 'last_peered': '2024-09-05T13:13:41.327431+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:13:41.327431+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 5332, 'log_dups_size': 0, 'ondisk_log_size': 5332, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21807104, 'num_objects': 5324, 'num_object_clones': 0, 'num_object_copies': 10648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1231, 'num_write_kb': 4924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 880}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:42.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:13:42.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5376", 'reported_seq': 5509, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:41.307397+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:13:41.307397+0000', 'last_peered': '2024-09-05T13:13:41.307397+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:13:41.307397+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 5376, 'log_dups_size': 0, 'ondisk_log_size': 5376, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22003712, 'num_objects': 5372, 'num_object_clones': 0, 'num_object_copies': 10744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1506, 'num_write_kb': 6024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 806}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:42.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:13:42.484 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5405", 'reported_seq': 5539, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:41.365443+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:13:41.365443+0000', 'last_peered': '2024-09-05T13:13:41.365443+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:13:41.365443+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 5405, 'log_dups_size': 0, 'ondisk_log_size': 5405, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22090211, 'num_objects': 5394, 'num_object_clones': 0, 'num_object_copies': 10788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5394, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2073, 'num_write_kb': 8292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 829}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:42.484 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:13:43.197 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:43.197 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/167742 objects degraded (29.276%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":150,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":83871,"data_bytes":343523830,"bytes_used":1129623552,"bytes_avail":288780668928,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":167742,"degraded_ratio":0.29275911816956995,"misplaced_objects":3993,"misplaced_total":167742,"misplaced_ratio":0.023804413921379261,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":77122,"recovering_keys_per_sec":0,"num_objects_recovered":226,"num_bytes_recovered":925696,"num_keys_recovered":0,"write_bytes_sec":2647751,"read_op_per_sec":0,"write_op_per_sec":646},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:13:43.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 70 20 85777 85757 4.78458 2.0625 0.0208757 0.0163169 2024-09-05T13:13:43.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:13:43.223 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:43.795 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:43.795 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:43.820 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-05T13:13:43.529517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":348008950,"num_objects":84966,"num_object_clones":0,"num_object_copies":169932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3870,"num_objects_unfound":0,"num_objects_dirty":84966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40765,"num_write_kb":163054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":2121747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85048,"ondisk_log_size":85048,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1113204,"kb_used_data":676676,"kb_used_omap":12,"kb_used_meta":436467,"kb_avail":282002316,"statfs":{"total":289910292480,"available":288770371584,"internally_reserved":0,"allocated":692916224,"data_stored":692093314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":446942214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,4,1,8,12,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28590080,"num_objects":6980,"num_object_clones":0,"num_object_copies":13960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-349,"num_objects_unfound":0,"num_objects_dirty":6980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6980,"num_write_kb":27920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":1429504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003395"},"pg_stats":[{"pgid":"2.f","version":"43'5240","reported_seq":5309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.216755+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:41.216755+0000","last_peered":"2024-09-05T13:13:41.216755+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:41.216755+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21463040,"num_objects":5240,"num_object_clones":0,"num_object_copies":10480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5240,"num_write_kb":20960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":819},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5298","reported_seq":5397,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:42.124315+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:42.124315+0000","last_peered":"2024-09-05T13:13:42.124315+0000","last_clean":"2024-09-05T13:13:42.124315+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:42.124315+0000","last_undegraded":"2024-09-05T13:13:42.124315+0000","last_fullsized":"2024-09-05T13:13:42.124315+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5298,"log_dups_size":0,"ondisk_log_size":5298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5355","reported_seq":5477,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:42.181506+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:42.181506+0000","last_peered":"2024-09-05T13:13:42.181506+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:42.181506+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5355,"log_dups_size":0,"ondisk_log_size":5355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21917696,"num_objects":5351,"num_object_clones":0,"num_object_copies":10702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3166,"num_write_kb":12664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":859}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5334","reported_seq":5398,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:42.152039+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:42.152039+0000","last_peered":"2024-09-05T13:13:42.152039+0000","last_clean":"2024-09-05T13:13:42.152039+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:42.152039+0000","last_undegraded":"2024-09-05T13:13:42.152039+0000","last_fullsized":"2024-09-05T13:13:42.152039+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5334,"log_dups_size":0,"ondisk_log_size":5334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5334,"num_write_kb":21336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5310","reported_seq":5439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:41.340295+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:41.340295+0000","last_peered":"2024-09-05T13:13:41.340295+0000","last_clean":"2024-09-05T13:13:41.340295+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:41.340295+0000","last_undegraded":"2024-09-05T13:13:41.340295+0000","last_fullsized":"2024-09-05T13:13:41.340295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21721088,"num_objects":5303,"num_object_clones":0,"num_object_copies":10606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":7400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5318","reported_seq":5451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.290704+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:41.290704+0000","last_peered":"2024-09-05T13:13:41.290704+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:41.290704+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5318,"log_dups_size":0,"ondisk_log_size":5318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":862},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5221","reported_seq":5355,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.370820+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:41.370820+0000","last_peered":"2024-09-05T13:13:41.370820+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:41.370820+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5221,"log_dups_size":0,"ondisk_log_size":5221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21360640,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1540,"num_write_kb":6160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":779},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5367","reported_seq":5493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:41.348839+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:41.348839+0000","last_peered":"2024-09-05T13:13:41.348839+0000","last_clean":"2024-09-05T13:13:41.348839+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:41.348839+0000","last_undegraded":"2024-09-05T13:13:41.348839+0000","last_fullsized":"2024-09-05T13:13:41.348839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5367,"log_dups_size":0,"ondisk_log_size":5367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21975040,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":7908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5335","reported_seq":5464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:42.217026+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:42.217026+0000","last_peered":"2024-09-05T13:13:42.217026+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:42.217026+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2989,"num_write_kb":11956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":877}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5302","reported_seq":5434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.316355+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:41.316355+0000","last_peered":"2024-09-05T13:13:41.316355+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:41.316355+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684224,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":6676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5292","reported_seq":6433,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:42.232949+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:42.232949+0000","last_peered":"2024-09-05T13:13:42.232949+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:42.232949+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5292,"log_dups_size":0,"ondisk_log_size":5292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3870,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2887,"num_write_kb":11548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":2109440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3870},{"shards":"2","objects":1415}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5283","reported_seq":5400,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:42.214308+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:42.214308+0000","last_peered":"2024-09-05T13:13:42.214308+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:42.214308+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5283,"log_dups_size":0,"ondisk_log_size":5283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21614592,"num_objects":5277,"num_object_clones":0,"num_object_copies":10554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2792,"num_write_kb":11168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":890}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5277","reported_seq":5412,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.351239+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:41.351239+0000","last_peered":"2024-09-05T13:13:41.351239+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:41.351239+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5277,"log_dups_size":0,"ondisk_log_size":5277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1428,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5332","reported_seq":5514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.327431+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:41.327431+0000","last_peered":"2024-09-05T13:13:41.327431+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:41.327431+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5332,"log_dups_size":0,"ondisk_log_size":5332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":880},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5376","reported_seq":5509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.307397+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:41.307397+0000","last_peered":"2024-09-05T13:13:41.307397+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:41.307397+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5376,"log_dups_size":0,"ondisk_log_size":5376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":806},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5405","reported_seq":5539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.365443+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:41.365443+0000","last_peered":"2024-09-05T13:13:41.365443+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:41.365443+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22090211,"num_objects":5394,"num_object_clones":0,"num_object_copies":10788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2073,"num_write_kb":8292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":829},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":348008931,"num_objects":84964,"num_object_clones":0,"num_object_copies":169928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3870,"num_objects_unfound":0,"num_objects_dirty":84964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40763,"num_write_kb":163052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":2121728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":692985856,"data_stored":692895531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85045,"ondisk_log_size":85045,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475904,"kb_used_data":313712,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93895936,"statfs":{"total":96636764160,"available":96149438464,"internally_reserved":0,"allocated":321241088,"data_stored":320995998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":166063789},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,3,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505172,"kb_used_data":324996,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93866668,"statfs":{"total":96636764160,"available":96119468032,"internally_reserved":0,"allocated":332795904,"data_stored":332546737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132128,"kb_used_data":37968,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94239712,"statfs":{"total":96636764160,"available":96501465088,"internally_reserved":0,"allocated":38879232,"data_stored":38550579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":333352960,"data_stored":333349347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":38690816,"data_stored":38607717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320942080,"data_stored":320938467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:44.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 71 20 86337 86317 4.74801 2.1875 0.052017 0.016446 2024-09-05T13:13:45.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 72 20 86937 86917 4.71461 2.34375 0.0323795 0.016563 2024-09-05T13:13:46.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 73 19 87549 87530 4.68282 2.39453 0.0229451 0.0166734 2024-09-05T13:13:46.822 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:47.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 74 20 88121 88101 4.64968 2.23047 0.0388346 0.0167939 2024-09-05T13:13:47.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:47.379 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:47.402 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-05T13:13:45.529957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":348008950,"num_objects":84966,"num_object_clones":0,"num_object_copies":169932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3870,"num_objects_unfound":0,"num_objects_dirty":84966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40765,"num_write_kb":163054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":2121747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85048,"ondisk_log_size":85048,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1113948,"kb_used_data":685736,"kb_used_omap":12,"kb_used_meta":428147,"kb_avail":282001572,"statfs":{"total":289910292480,"available":288769609728,"internally_reserved":0,"allocated":702193664,"data_stored":701370754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":438422534},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,3,4,8,11,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24088576,"num_objects":5881,"num_object_clones":0,"num_object_copies":11762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-232,"num_objects_unfound":0,"num_objects_dirty":5881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5881,"num_write_kb":23524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":950272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003414"},"pg_stats":[{"pgid":"2.f","version":"43'5240","reported_seq":5309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.216755+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:41.216755+0000","last_peered":"2024-09-05T13:13:41.216755+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:41.216755+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21463040,"num_objects":5240,"num_object_clones":0,"num_object_copies":10480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5240,"num_write_kb":20960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":819},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5298","reported_seq":5397,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:42.124315+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:42.124315+0000","last_peered":"2024-09-05T13:13:42.124315+0000","last_clean":"2024-09-05T13:13:42.124315+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:42.124315+0000","last_undegraded":"2024-09-05T13:13:42.124315+0000","last_fullsized":"2024-09-05T13:13:42.124315+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5298,"log_dups_size":0,"ondisk_log_size":5298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21676032,"num_objects":5292,"num_object_clones":0,"num_object_copies":10584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3308,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5355","reported_seq":5477,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:42.181506+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:42.181506+0000","last_peered":"2024-09-05T13:13:42.181506+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:42.181506+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5355,"log_dups_size":0,"ondisk_log_size":5355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21917696,"num_objects":5351,"num_object_clones":0,"num_object_copies":10702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3166,"num_write_kb":12664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":859}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5334","reported_seq":5398,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:42.152039+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:42.152039+0000","last_peered":"2024-09-05T13:13:42.152039+0000","last_clean":"2024-09-05T13:13:42.152039+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:42.152039+0000","last_undegraded":"2024-09-05T13:13:42.152039+0000","last_fullsized":"2024-09-05T13:13:42.152039+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5334,"log_dups_size":0,"ondisk_log_size":5334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5334,"num_write_kb":21336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5310","reported_seq":5439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:41.340295+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:41.340295+0000","last_peered":"2024-09-05T13:13:41.340295+0000","last_clean":"2024-09-05T13:13:41.340295+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:41.340295+0000","last_undegraded":"2024-09-05T13:13:41.340295+0000","last_fullsized":"2024-09-05T13:13:41.340295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21721088,"num_objects":5303,"num_object_clones":0,"num_object_copies":10606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":7400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5318","reported_seq":5451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.290704+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:41.290704+0000","last_peered":"2024-09-05T13:13:41.290704+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:41.290704+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5318,"log_dups_size":0,"ondisk_log_size":5318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":862},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5221","reported_seq":5355,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.370820+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:41.370820+0000","last_peered":"2024-09-05T13:13:41.370820+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:41.370820+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5221,"log_dups_size":0,"ondisk_log_size":5221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21360640,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1540,"num_write_kb":6160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":779},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5367","reported_seq":5493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:41.348839+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:41.348839+0000","last_peered":"2024-09-05T13:13:41.348839+0000","last_clean":"2024-09-05T13:13:41.348839+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:41.348839+0000","last_undegraded":"2024-09-05T13:13:41.348839+0000","last_fullsized":"2024-09-05T13:13:41.348839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5367,"log_dups_size":0,"ondisk_log_size":5367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21975040,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1977,"num_write_kb":7908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5335","reported_seq":5464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:42.217026+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:42.217026+0000","last_peered":"2024-09-05T13:13:42.217026+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:42.217026+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2989,"num_write_kb":11956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":877}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5302","reported_seq":5434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.316355+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:41.316355+0000","last_peered":"2024-09-05T13:13:41.316355+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:41.316355+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684224,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":6676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5292","reported_seq":6433,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:42.232949+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:42.232949+0000","last_peered":"2024-09-05T13:13:42.232949+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:42.232949+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5292,"log_dups_size":0,"ondisk_log_size":5292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3870,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2887,"num_write_kb":11548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":2109440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3870},{"shards":"2","objects":1415}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5283","reported_seq":5400,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:42.214308+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:42.214308+0000","last_peered":"2024-09-05T13:13:42.214308+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:42.214308+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5283,"log_dups_size":0,"ondisk_log_size":5283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21614592,"num_objects":5277,"num_object_clones":0,"num_object_copies":10554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2792,"num_write_kb":11168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":890}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5277","reported_seq":5412,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.351239+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:41.351239+0000","last_peered":"2024-09-05T13:13:41.351239+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:41.351239+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5277,"log_dups_size":0,"ondisk_log_size":5277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1428,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5332","reported_seq":5514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.327431+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:41.327431+0000","last_peered":"2024-09-05T13:13:41.327431+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:41.327431+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5332,"log_dups_size":0,"ondisk_log_size":5332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":880},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5376","reported_seq":5509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.307397+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:41.307397+0000","last_peered":"2024-09-05T13:13:41.307397+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:41.307397+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5376,"log_dups_size":0,"ondisk_log_size":5376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22003712,"num_objects":5372,"num_object_clones":0,"num_object_copies":10744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":806},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5405","reported_seq":5539,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:41.365443+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:41.365443+0000","last_peered":"2024-09-05T13:13:41.365443+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:41.365443+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22090211,"num_objects":5394,"num_object_clones":0,"num_object_copies":10788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2073,"num_write_kb":8292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":829},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":348008931,"num_objects":84964,"num_object_clones":0,"num_object_copies":169928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3870,"num_objects_unfound":0,"num_objects_dirty":84964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40763,"num_write_kb":163052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":518,"num_bytes_recovered":2121728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":702320640,"data_stored":702230315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85045,"ondisk_log_size":85045,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475904,"kb_used_data":313712,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93895936,"statfs":{"total":96636764160,"available":96149438464,"internally_reserved":0,"allocated":321241088,"data_stored":320995998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":166063789},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,3,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505172,"kb_used_data":324996,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93866668,"statfs":{"total":96636764160,"available":96119468032,"internally_reserved":0,"allocated":332795904,"data_stored":332546737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132872,"kb_used_data":47028,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94238968,"statfs":{"total":96636764160,"available":96500703232,"internally_reserved":0,"allocated":48156672,"data_stored":47828019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":87879343},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":333352960,"data_stored":333349347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":48025600,"data_stored":47942501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":320942080,"data_stored":320938467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:47.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:47.953 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:47.955 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:47.981 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-05T13:13:47.530374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":359768566,"num_objects":87837,"num_object_clones":0,"num_object_copies":175674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":87837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43636,"num_write_kb":174538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":2584595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87919,"ondisk_log_size":87919,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1158640,"kb_used_data":699900,"kb_used_omap":12,"kb_used_meta":458675,"kb_avail":281956880,"statfs":{"total":289910292480,"available":288723845120,"internally_reserved":0,"allocated":716697600,"data_stored":715874690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":469683206},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,2,2,7,6,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-345,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8752,"num_write_kb":35008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003374"},"pg_stats":[{"pgid":"2.f","version":"43'5405","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.266975+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:46.266975+0000","last_peered":"2024-09-05T13:13:46.266975+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:46.266975+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22138880,"num_objects":5405,"num_object_clones":0,"num_object_copies":10810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5405,"num_write_kb":21620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":984},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5480","reported_seq":5579,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:47.270078+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:47.270078+0000","last_peered":"2024-09-05T13:13:47.270078+0000","last_clean":"2024-09-05T13:13:47.270078+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:47.270078+0000","last_undegraded":"2024-09-05T13:13:47.270078+0000","last_fullsized":"2024-09-05T13:13:47.270078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5480,"log_dups_size":0,"ondisk_log_size":5480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3490,"num_write_kb":13960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5532","reported_seq":5654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.179807+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:47.179807+0000","last_peered":"2024-09-05T13:13:47.179807+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:47.179807+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1036}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5540","reported_seq":5604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:47.209341+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:47.209341+0000","last_peered":"2024-09-05T13:13:47.209341+0000","last_clean":"2024-09-05T13:13:47.209341+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:47.209341+0000","last_undegraded":"2024-09-05T13:13:47.209341+0000","last_fullsized":"2024-09-05T13:13:47.209341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5540,"log_dups_size":0,"ondisk_log_size":5540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22691840,"num_objects":5540,"num_object_clones":0,"num_object_copies":11080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5483","reported_seq":5612,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:46.281656+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:46.281656+0000","last_peered":"2024-09-05T13:13:46.281656+0000","last_clean":"2024-09-05T13:13:46.281656+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:46.281656+0000","last_undegraded":"2024-09-05T13:13:46.281656+0000","last_fullsized":"2024-09-05T13:13:46.281656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5483,"log_dups_size":0,"ondisk_log_size":5483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22429696,"num_objects":5476,"num_object_clones":0,"num_object_copies":10952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2023,"num_write_kb":8092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5481","reported_seq":5614,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.344775+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:46.344775+0000","last_peered":"2024-09-05T13:13:46.344775+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:46.344775+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22429696,"num_objects":5476,"num_object_clones":0,"num_object_copies":10952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1936,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1025},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5421","reported_seq":5553,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.368987+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:46.368987+0000","last_peered":"2024-09-05T13:13:46.368987+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:46.368987+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5421,"log_dups_size":0,"ondisk_log_size":5421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22179840,"num_objects":5415,"num_object_clones":0,"num_object_copies":10830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1740,"num_write_kb":6960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":979},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5537","reported_seq":5663,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:46.293554+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:46.293554+0000","last_peered":"2024-09-05T13:13:46.293554+0000","last_clean":"2024-09-05T13:13:46.293554+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:46.293554+0000","last_undegraded":"2024-09-05T13:13:46.293554+0000","last_fullsized":"2024-09-05T13:13:46.293554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5537,"log_dups_size":0,"ondisk_log_size":5537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671360,"num_objects":5535,"num_object_clones":0,"num_object_copies":11070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2147,"num_write_kb":8588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5518","reported_seq":5647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.270480+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:47.270480+0000","last_peered":"2024-09-05T13:13:47.270480+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:47.270480+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3172,"num_write_kb":12688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5475","reported_seq":5607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.383007+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:46.383007+0000","last_peered":"2024-09-05T13:13:46.383007+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:46.383007+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22392832,"num_objects":5467,"num_object_clones":0,"num_object_copies":10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":7368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1013},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5460","reported_seq":6826,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.261024+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:47.261024+0000","last_peered":"2024-09-05T13:13:47.261024+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:47.261024+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5460,"log_dups_size":0,"ondisk_log_size":5460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3055,"num_write_kb":12220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":2572288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3757},{"shards":"2","objects":1696}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5464","reported_seq":5581,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.255611+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:47.255611+0000","last_peered":"2024-09-05T13:13:47.255611+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:47.255611+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22355968,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2973,"num_write_kb":11892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1071}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5467","reported_seq":5601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.327209+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:46.327209+0000","last_peered":"2024-09-05T13:13:46.327209+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:46.327209+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22384640,"num_objects":5465,"num_object_clones":0,"num_object_copies":10930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1618,"num_write_kb":6472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1046},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5510","reported_seq":5692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.377069+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:46.377069+0000","last_peered":"2024-09-05T13:13:46.377069+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:46.377069+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22536192,"num_objects":5502,"num_object_clones":0,"num_object_copies":11004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1409,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1058},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5560","reported_seq":5693,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.303061+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:46.303061+0000","last_peered":"2024-09-05T13:13:46.303061+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:46.303061+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5560,"log_dups_size":0,"ondisk_log_size":5560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22757376,"num_objects":5556,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1690,"num_write_kb":6760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":990},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5583","reported_seq":5713,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.388911+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:46.388911+0000","last_peered":"2024-09-05T13:13:46.388911+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:46.388911+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5583,"log_dups_size":0,"ondisk_log_size":5583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22819299,"num_objects":5572,"num_object_clones":0,"num_object_copies":11144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2251,"num_write_kb":9004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1007},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":359768547,"num_objects":87835,"num_object_clones":0,"num_object_copies":175670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":87835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43634,"num_write_kb":174536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":2584576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":717115392,"data_stored":717025067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87916,"ondisk_log_size":87916,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503832,"kb_used_data":319432,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93868008,"statfs":{"total":96636764160,"available":96120840192,"internally_reserved":0,"allocated":327098368,"data_stored":326853278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521936,"kb_used_data":333440,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93849904,"statfs":{"total":96636764160,"available":96102301696,"internally_reserved":0,"allocated":341442560,"data_stored":341193393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":192999082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,0,1,0,1,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132872,"kb_used_data":47028,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94238968,"statfs":{"total":96636764160,"available":96500703232,"internally_reserved":0,"allocated":48156672,"data_stored":47828019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":87879343},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342233088,"data_stored":342229475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":48025600,"data_stored":47942501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326856704,"data_stored":326853091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:47.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:13:47.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5405", 'reported_seq': 5474, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:46.266975+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:13:46.266975+0000', 'last_peered': '2024-09-05T13:13:46.266975+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:13:46.266975+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 5405, 'log_dups_size': 0, 'ondisk_log_size': 5405, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22138880, 'num_objects': 5405, 'num_object_clones': 0, 'num_object_copies': 10810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5405, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5405, 'num_write_kb': 21620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 984}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:47.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:13:47.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5532", 'reported_seq': 5654, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:47.179807+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:13:47.179807+0000', 'last_peered': '2024-09-05T13:13:47.179807+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:13:47.179807+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 5532, 'log_dups_size': 0, 'ondisk_log_size': 5532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22642688, 'num_objects': 5528, 'num_object_clones': 0, 'num_object_copies': 11056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3343, 'num_write_kb': 13372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 1036}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:47.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:13:47.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5481", 'reported_seq': 5614, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:46.344775+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:13:46.344775+0000', 'last_peered': '2024-09-05T13:13:46.344775+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:13:46.344775+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 5481, 'log_dups_size': 0, 'ondisk_log_size': 5481, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22429696, 'num_objects': 5476, 'num_object_clones': 0, 'num_object_copies': 10952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1936, 'num_write_kb': 7744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1025}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:47.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:13:47.984 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5421", 'reported_seq': 5553, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:46.368987+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:13:46.368987+0000', 'last_peered': '2024-09-05T13:13:46.368987+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:13:46.368987+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 5421, 'log_dups_size': 0, 'ondisk_log_size': 5421, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22179840, 'num_objects': 5415, 'num_object_clones': 0, 'num_object_copies': 10830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1740, 'num_write_kb': 6960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 979}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:47.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:13:47.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5518", 'reported_seq': 5647, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:47.270480+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:13:47.270480+0000', 'last_peered': '2024-09-05T13:13:47.270480+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:13:47.270480+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 5518, 'log_dups_size': 0, 'ondisk_log_size': 5518, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22573056, 'num_objects': 5511, 'num_object_clones': 0, 'num_object_copies': 11022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5511, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3172, 'num_write_kb': 12688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 1060}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:47.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:13:47.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5475", 'reported_seq': 5607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:46.383007+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:13:46.383007+0000', 'last_peered': '2024-09-05T13:13:46.383007+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:13:46.383007+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 5475, 'log_dups_size': 0, 'ondisk_log_size': 5475, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22392832, 'num_objects': 5467, 'num_object_clones': 0, 'num_object_copies': 10934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1842, 'num_write_kb': 7368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1013}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:47.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:13:47.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5460", 'reported_seq': 6826, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:47.261024+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:13:47.261024+0000', 'last_peered': '2024-09-05T13:13:47.261024+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:13:47.261024+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 5460, 'log_dups_size': 0, 'ondisk_log_size': 5460, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22343680, 'num_objects': 5455, 'num_object_clones': 0, 'num_object_copies': 10910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3757, 'num_objects_unfound': 0, 'num_objects_dirty': 5455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3055, 'num_write_kb': 12220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 628, 'num_bytes_recovered': 2572288, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3757}, {'shards': '2', 'objects': 1696}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:47.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:13:47.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5464", 'reported_seq': 5581, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:47.255611+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:13:47.255611+0000', 'last_peered': '2024-09-05T13:13:47.255611+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:13:47.255611+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 5464, 'log_dups_size': 0, 'ondisk_log_size': 5464, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22355968, 'num_objects': 5458, 'num_object_clones': 0, 'num_object_copies': 10916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2973, 'num_write_kb': 11892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 1071}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:47.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:13:47.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5467", 'reported_seq': 5601, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:46.327209+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:13:46.327209+0000', 'last_peered': '2024-09-05T13:13:46.327209+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:13:46.327209+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 5467, 'log_dups_size': 0, 'ondisk_log_size': 5467, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22384640, 'num_objects': 5465, 'num_object_clones': 0, 'num_object_copies': 10930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1618, 'num_write_kb': 6472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1046}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:47.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:13:47.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5510", 'reported_seq': 5692, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:46.377069+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:13:46.377069+0000', 'last_peered': '2024-09-05T13:13:46.377069+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:13:46.377069+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 5510, 'log_dups_size': 0, 'ondisk_log_size': 5510, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22536192, 'num_objects': 5502, 'num_object_clones': 0, 'num_object_copies': 11004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1409, 'num_write_kb': 5636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1058}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:47.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:13:47.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5560", 'reported_seq': 5693, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:46.303061+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:13:46.303061+0000', 'last_peered': '2024-09-05T13:13:46.303061+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:13:46.303061+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 5560, 'log_dups_size': 0, 'ondisk_log_size': 5560, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22757376, 'num_objects': 5556, 'num_object_clones': 0, 'num_object_copies': 11112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5556, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1690, 'num_write_kb': 6760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 990}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:47.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:13:47.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5583", 'reported_seq': 5713, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:46.388911+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:13:46.388911+0000', 'last_peered': '2024-09-05T13:13:46.388911+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:13:46.388911+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 5583, 'log_dups_size': 0, 'ondisk_log_size': 5583, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22819299, 'num_objects': 5572, 'num_object_clones': 0, 'num_object_copies': 11144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2251, 'num_write_kb': 9004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1007}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:47.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:13:48.200 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 75 20 88653 88633 4.6154 2.07812 0.0231474 0.0169186 2024-09-05T13:13:48.681 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:48.681 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/175674 objects degraded (27.954%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":155,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":87837,"data_bytes":359768566,"bytes_used":1186447360,"bytes_avail":288723845120,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":175674,"degraded_ratio":0.27954051254027346,"misplaced_objects":3757,"misplaced_total":175674,"misplaced_ratio":0.021386203991484228,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":117726,"recovering_keys_per_sec":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"write_bytes_sec":2986509,"read_op_per_sec":0,"write_op_per_sec":729},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:13:48.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:13:48.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:49.200 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 76 20 89179 89159 4.5817 2.05469 0.0162478 0.0170415 2024-09-05T13:13:49.273 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:49.274 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:49.296 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-05T13:13:47.530374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":359768566,"num_objects":87837,"num_object_clones":0,"num_object_copies":175674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":87837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43636,"num_write_kb":174538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":2584595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87919,"ondisk_log_size":87919,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1158640,"kb_used_data":699900,"kb_used_omap":12,"kb_used_meta":458675,"kb_avail":281956880,"statfs":{"total":289910292480,"available":288723845120,"internally_reserved":0,"allocated":716697600,"data_stored":715874690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":469683206},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,2,2,7,6,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-345,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8752,"num_write_kb":35008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003374"},"pg_stats":[{"pgid":"2.f","version":"43'5405","reported_seq":5474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.266975+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:46.266975+0000","last_peered":"2024-09-05T13:13:46.266975+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:46.266975+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5405,"log_dups_size":0,"ondisk_log_size":5405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22138880,"num_objects":5405,"num_object_clones":0,"num_object_copies":10810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5405,"num_write_kb":21620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":984},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5480","reported_seq":5579,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:47.270078+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:47.270078+0000","last_peered":"2024-09-05T13:13:47.270078+0000","last_clean":"2024-09-05T13:13:47.270078+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:47.270078+0000","last_undegraded":"2024-09-05T13:13:47.270078+0000","last_fullsized":"2024-09-05T13:13:47.270078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5480,"log_dups_size":0,"ondisk_log_size":5480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3490,"num_write_kb":13960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5532","reported_seq":5654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.179807+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:47.179807+0000","last_peered":"2024-09-05T13:13:47.179807+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:47.179807+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1036}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5540","reported_seq":5604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:47.209341+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:47.209341+0000","last_peered":"2024-09-05T13:13:47.209341+0000","last_clean":"2024-09-05T13:13:47.209341+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:47.209341+0000","last_undegraded":"2024-09-05T13:13:47.209341+0000","last_fullsized":"2024-09-05T13:13:47.209341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5540,"log_dups_size":0,"ondisk_log_size":5540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22691840,"num_objects":5540,"num_object_clones":0,"num_object_copies":11080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5483","reported_seq":5612,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:46.281656+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:46.281656+0000","last_peered":"2024-09-05T13:13:46.281656+0000","last_clean":"2024-09-05T13:13:46.281656+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:46.281656+0000","last_undegraded":"2024-09-05T13:13:46.281656+0000","last_fullsized":"2024-09-05T13:13:46.281656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5483,"log_dups_size":0,"ondisk_log_size":5483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22429696,"num_objects":5476,"num_object_clones":0,"num_object_copies":10952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2023,"num_write_kb":8092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5481","reported_seq":5614,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.344775+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:46.344775+0000","last_peered":"2024-09-05T13:13:46.344775+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:46.344775+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22429696,"num_objects":5476,"num_object_clones":0,"num_object_copies":10952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1936,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1025},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5421","reported_seq":5553,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.368987+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:46.368987+0000","last_peered":"2024-09-05T13:13:46.368987+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:46.368987+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5421,"log_dups_size":0,"ondisk_log_size":5421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22179840,"num_objects":5415,"num_object_clones":0,"num_object_copies":10830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1740,"num_write_kb":6960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":979},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5537","reported_seq":5663,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:46.293554+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:46.293554+0000","last_peered":"2024-09-05T13:13:46.293554+0000","last_clean":"2024-09-05T13:13:46.293554+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:46.293554+0000","last_undegraded":"2024-09-05T13:13:46.293554+0000","last_fullsized":"2024-09-05T13:13:46.293554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5537,"log_dups_size":0,"ondisk_log_size":5537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671360,"num_objects":5535,"num_object_clones":0,"num_object_copies":11070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2147,"num_write_kb":8588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5518","reported_seq":5647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.270480+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:47.270480+0000","last_peered":"2024-09-05T13:13:47.270480+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:47.270480+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3172,"num_write_kb":12688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5475","reported_seq":5607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.383007+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:46.383007+0000","last_peered":"2024-09-05T13:13:46.383007+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:46.383007+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22392832,"num_objects":5467,"num_object_clones":0,"num_object_copies":10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":7368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1013},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5460","reported_seq":6826,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.261024+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:47.261024+0000","last_peered":"2024-09-05T13:13:47.261024+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:47.261024+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5460,"log_dups_size":0,"ondisk_log_size":5460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3055,"num_write_kb":12220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":2572288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3757},{"shards":"2","objects":1696}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5464","reported_seq":5581,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.255611+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:47.255611+0000","last_peered":"2024-09-05T13:13:47.255611+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:47.255611+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22355968,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2973,"num_write_kb":11892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1071}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5467","reported_seq":5601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.327209+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:46.327209+0000","last_peered":"2024-09-05T13:13:46.327209+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:46.327209+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22384640,"num_objects":5465,"num_object_clones":0,"num_object_copies":10930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1618,"num_write_kb":6472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1046},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5510","reported_seq":5692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.377069+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:46.377069+0000","last_peered":"2024-09-05T13:13:46.377069+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:46.377069+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22536192,"num_objects":5502,"num_object_clones":0,"num_object_copies":11004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1409,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1058},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5560","reported_seq":5693,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.303061+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:46.303061+0000","last_peered":"2024-09-05T13:13:46.303061+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:46.303061+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5560,"log_dups_size":0,"ondisk_log_size":5560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22757376,"num_objects":5556,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1690,"num_write_kb":6760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":990},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5583","reported_seq":5713,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:46.388911+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:46.388911+0000","last_peered":"2024-09-05T13:13:46.388911+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:46.388911+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5583,"log_dups_size":0,"ondisk_log_size":5583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22819299,"num_objects":5572,"num_object_clones":0,"num_object_copies":11144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2251,"num_write_kb":9004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1007},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":359768547,"num_objects":87835,"num_object_clones":0,"num_object_copies":175670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":87835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43634,"num_write_kb":174536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":2584576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":717115392,"data_stored":717025067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87916,"ondisk_log_size":87916,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503832,"kb_used_data":319432,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93868008,"statfs":{"total":96636764160,"available":96120840192,"internally_reserved":0,"allocated":327098368,"data_stored":326853278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521936,"kb_used_data":333440,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93849904,"statfs":{"total":96636764160,"available":96102301696,"internally_reserved":0,"allocated":341442560,"data_stored":341193393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":192999082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,0,1,0,1,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132872,"kb_used_data":47028,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94238968,"statfs":{"total":96636764160,"available":96500703232,"internally_reserved":0,"allocated":48156672,"data_stored":47828019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":87879343},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342233088,"data_stored":342229475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":48025600,"data_stored":47942501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326856704,"data_stored":326853091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:50.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 77 20 89767 89747 4.55203 2.29688 0.045033 0.0171535 2024-09-05T13:13:51.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 78 20 90372 90352 4.52377 2.36328 0.0184141 0.0172626 2024-09-05T13:13:52.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 79 20 90964 90944 4.49578 2.3125 0.0310263 0.0173707 2024-09-05T13:13:52.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:52.852 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:52.876 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-05T13:13:51.531748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":367153654,"num_objects":89640,"num_object_clones":0,"num_object_copies":179280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":89640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45439,"num_write_kb":181750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":2584595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89722,"ondisk_log_size":89722,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1172312,"kb_used_data":717736,"kb_used_omap":12,"kb_used_meta":454515,"kb_avail":281943208,"statfs":{"total":289910292480,"available":288709844992,"internally_reserved":0,"allocated":734961664,"data_stored":734138754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":465423366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,3,9,14,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31051776,"num_objects":7581,"num_object_clones":0,"num_object_copies":15162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-236,"num_objects_unfound":0,"num_objects_dirty":7581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7581,"num_write_kb":30324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003711"},"pg_stats":[{"pgid":"2.f","version":"43'5587","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.375454+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:51.375454+0000","last_peered":"2024-09-05T13:13:51.375454+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:51.375454+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5587,"log_dups_size":0,"ondisk_log_size":5587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884352,"num_objects":5587,"num_object_clones":0,"num_object_copies":11174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5587,"num_write_kb":22348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1166},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5480","reported_seq":5579,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:47.270078+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:47.270078+0000","last_peered":"2024-09-05T13:13:47.270078+0000","last_clean":"2024-09-05T13:13:47.270078+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:47.270078+0000","last_undegraded":"2024-09-05T13:13:47.270078+0000","last_fullsized":"2024-09-05T13:13:47.270078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5480,"log_dups_size":0,"ondisk_log_size":5480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3490,"num_write_kb":13960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5532","reported_seq":5654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.179807+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:47.179807+0000","last_peered":"2024-09-05T13:13:47.179807+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:47.179807+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1036}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5540","reported_seq":5604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:47.209341+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:47.209341+0000","last_peered":"2024-09-05T13:13:47.209341+0000","last_clean":"2024-09-05T13:13:47.209341+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:47.209341+0000","last_undegraded":"2024-09-05T13:13:47.209341+0000","last_fullsized":"2024-09-05T13:13:47.209341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5540,"log_dups_size":0,"ondisk_log_size":5540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22691840,"num_objects":5540,"num_object_clones":0,"num_object_copies":11080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5671","reported_seq":5800,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:51.410863+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:51.410863+0000","last_peered":"2024-09-05T13:13:51.410863+0000","last_clean":"2024-09-05T13:13:51.410863+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:51.410863+0000","last_undegraded":"2024-09-05T13:13:51.410863+0000","last_fullsized":"2024-09-05T13:13:51.410863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5671,"log_dups_size":0,"ondisk_log_size":5671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2211,"num_write_kb":8844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5669","reported_seq":5799,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.411192+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:51.411192+0000","last_peered":"2024-09-05T13:13:51.411192+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:51.411192+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5669,"log_dups_size":0,"ondisk_log_size":5669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2124,"num_write_kb":8496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5605","reported_seq":5739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.360190+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:51.360190+0000","last_peered":"2024-09-05T13:13:51.360190+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:51.360190+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933504,"num_objects":5599,"num_object_clones":0,"num_object_copies":11198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1924,"num_write_kb":7696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1163},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5725","reported_seq":5851,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:51.393687+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:51.393687+0000","last_peered":"2024-09-05T13:13:51.393687+0000","last_clean":"2024-09-05T13:13:51.393687+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:51.393687+0000","last_undegraded":"2024-09-05T13:13:51.393687+0000","last_fullsized":"2024-09-05T13:13:51.393687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5725,"log_dups_size":0,"ondisk_log_size":5725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23441408,"num_objects":5723,"num_object_clones":0,"num_object_copies":11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2335,"num_write_kb":9340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5518","reported_seq":5647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.270480+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:47.270480+0000","last_peered":"2024-09-05T13:13:47.270480+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:47.270480+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3172,"num_write_kb":12688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5667","reported_seq":5799,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.407653+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:51.407653+0000","last_peered":"2024-09-05T13:13:51.407653+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:51.407653+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5667,"log_dups_size":0,"ondisk_log_size":5667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23179264,"num_objects":5659,"num_object_clones":0,"num_object_copies":11318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2034,"num_write_kb":8136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1205},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5460","reported_seq":6826,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.261024+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:47.261024+0000","last_peered":"2024-09-05T13:13:47.261024+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:47.261024+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5460,"log_dups_size":0,"ondisk_log_size":5460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3055,"num_write_kb":12220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":2572288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3757},{"shards":"2","objects":1696}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5464","reported_seq":5581,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.255611+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:47.255611+0000","last_peered":"2024-09-05T13:13:47.255611+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:47.255611+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22355968,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2973,"num_write_kb":11892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1071}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5619","reported_seq":5754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.311965+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:51.311965+0000","last_peered":"2024-09-05T13:13:51.311965+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:51.311965+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5619,"log_dups_size":0,"ondisk_log_size":5619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1770,"num_write_kb":7080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1198},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5680","reported_seq":5862,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.403751+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:51.403751+0000","last_peered":"2024-09-05T13:13:51.403751+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:51.403751+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5680,"log_dups_size":0,"ondisk_log_size":5680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1228},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5730","reported_seq":5863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.359772+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:51.359772+0000","last_peered":"2024-09-05T13:13:51.359772+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:51.359772+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1860,"num_write_kb":7440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1160},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5772","reported_seq":5906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.372404+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:51.372404+0000","last_peered":"2024-09-05T13:13:51.372404+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:51.372404+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5772,"log_dups_size":0,"ondisk_log_size":5772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23593443,"num_objects":5761,"num_object_clones":0,"num_object_copies":11522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":367153635,"num_objects":89638,"num_object_clones":0,"num_object_copies":179276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":89638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45437,"num_write_kb":181748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":2584576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":735207424,"data_stored":735117099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89719,"ondisk_log_size":89719,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503832,"kb_used_data":319432,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93868008,"statfs":{"total":96636764160,"available":96120840192,"internally_reserved":0,"allocated":327098368,"data_stored":326853278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518392,"kb_used_data":342376,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93853448,"statfs":{"total":96636764160,"available":96105930752,"internally_reserved":0,"allocated":350593024,"data_stored":350343857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,0,4,9,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150088,"kb_used_data":55928,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94221752,"statfs":{"total":96636764160,"available":96483074048,"internally_reserved":0,"allocated":57270272,"data_stored":56941619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351178752,"data_stored":351175139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57171968,"data_stored":57088869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326856704,"data_stored":326853091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:52.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:53.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:13:53.203698+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0174816 2024-09-05T13:13:53.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:13:53.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 80 20 91503 91483 4.4659 2.10547 0.108389 0.0174816 2024-09-05T13:13:53.438 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:53.441 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:53.463 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-05T13:13:51.531748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":367153654,"num_objects":89640,"num_object_clones":0,"num_object_copies":179280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":89640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45439,"num_write_kb":181750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":632,"num_bytes_recovered":2584595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89722,"ondisk_log_size":89722,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1172312,"kb_used_data":717736,"kb_used_omap":12,"kb_used_meta":454515,"kb_avail":281943208,"statfs":{"total":289910292480,"available":288709844992,"internally_reserved":0,"allocated":734961664,"data_stored":734138754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":465423366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,5,3,9,14,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31051776,"num_objects":7581,"num_object_clones":0,"num_object_copies":15162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-236,"num_objects_unfound":0,"num_objects_dirty":7581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7581,"num_write_kb":30324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003711"},"pg_stats":[{"pgid":"2.f","version":"43'5587","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.375454+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:51.375454+0000","last_peered":"2024-09-05T13:13:51.375454+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:51.375454+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5587,"log_dups_size":0,"ondisk_log_size":5587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884352,"num_objects":5587,"num_object_clones":0,"num_object_copies":11174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5587,"num_write_kb":22348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1166},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5480","reported_seq":5579,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:47.270078+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:47.270078+0000","last_peered":"2024-09-05T13:13:47.270078+0000","last_clean":"2024-09-05T13:13:47.270078+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:47.270078+0000","last_undegraded":"2024-09-05T13:13:47.270078+0000","last_fullsized":"2024-09-05T13:13:47.270078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5480,"log_dups_size":0,"ondisk_log_size":5480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3490,"num_write_kb":13960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5532","reported_seq":5654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.179807+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:47.179807+0000","last_peered":"2024-09-05T13:13:47.179807+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:47.179807+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3343,"num_write_kb":13372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1036}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5540","reported_seq":5604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:47.209341+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:47.209341+0000","last_peered":"2024-09-05T13:13:47.209341+0000","last_clean":"2024-09-05T13:13:47.209341+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:47.209341+0000","last_undegraded":"2024-09-05T13:13:47.209341+0000","last_fullsized":"2024-09-05T13:13:47.209341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5540,"log_dups_size":0,"ondisk_log_size":5540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22691840,"num_objects":5540,"num_object_clones":0,"num_object_copies":11080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5671","reported_seq":5800,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:51.410863+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:51.410863+0000","last_peered":"2024-09-05T13:13:51.410863+0000","last_clean":"2024-09-05T13:13:51.410863+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:51.410863+0000","last_undegraded":"2024-09-05T13:13:51.410863+0000","last_fullsized":"2024-09-05T13:13:51.410863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5671,"log_dups_size":0,"ondisk_log_size":5671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2211,"num_write_kb":8844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5669","reported_seq":5799,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.411192+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:51.411192+0000","last_peered":"2024-09-05T13:13:51.411192+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:51.411192+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5669,"log_dups_size":0,"ondisk_log_size":5669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2124,"num_write_kb":8496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5605","reported_seq":5739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.360190+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:51.360190+0000","last_peered":"2024-09-05T13:13:51.360190+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:51.360190+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933504,"num_objects":5599,"num_object_clones":0,"num_object_copies":11198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1924,"num_write_kb":7696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1163},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5725","reported_seq":5851,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:51.393687+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:51.393687+0000","last_peered":"2024-09-05T13:13:51.393687+0000","last_clean":"2024-09-05T13:13:51.393687+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:51.393687+0000","last_undegraded":"2024-09-05T13:13:51.393687+0000","last_fullsized":"2024-09-05T13:13:51.393687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5725,"log_dups_size":0,"ondisk_log_size":5725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23441408,"num_objects":5723,"num_object_clones":0,"num_object_copies":11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2335,"num_write_kb":9340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5518","reported_seq":5647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.270480+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:47.270480+0000","last_peered":"2024-09-05T13:13:47.270480+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:47.270480+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5518,"log_dups_size":0,"ondisk_log_size":5518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22573056,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3172,"num_write_kb":12688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1060}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5667","reported_seq":5799,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.407653+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:51.407653+0000","last_peered":"2024-09-05T13:13:51.407653+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:51.407653+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5667,"log_dups_size":0,"ondisk_log_size":5667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23179264,"num_objects":5659,"num_object_clones":0,"num_object_copies":11318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2034,"num_write_kb":8136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1205},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5460","reported_seq":6826,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.261024+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:47.261024+0000","last_peered":"2024-09-05T13:13:47.261024+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:47.261024+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5460,"log_dups_size":0,"ondisk_log_size":5460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3055,"num_write_kb":12220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":2572288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3757},{"shards":"2","objects":1696}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5464","reported_seq":5581,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:47.255611+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:47.255611+0000","last_peered":"2024-09-05T13:13:47.255611+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:47.255611+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22355968,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2973,"num_write_kb":11892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1071}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5619","reported_seq":5754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.311965+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:51.311965+0000","last_peered":"2024-09-05T13:13:51.311965+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:51.311965+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5619,"log_dups_size":0,"ondisk_log_size":5619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1770,"num_write_kb":7080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1198},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5680","reported_seq":5862,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.403751+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:51.403751+0000","last_peered":"2024-09-05T13:13:51.403751+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:51.403751+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5680,"log_dups_size":0,"ondisk_log_size":5680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1228},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5730","reported_seq":5863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.359772+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:51.359772+0000","last_peered":"2024-09-05T13:13:51.359772+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:51.359772+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1860,"num_write_kb":7440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1160},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5772","reported_seq":5906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.372404+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:51.372404+0000","last_peered":"2024-09-05T13:13:51.372404+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:51.372404+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5772,"log_dups_size":0,"ondisk_log_size":5772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23593443,"num_objects":5761,"num_object_clones":0,"num_object_copies":11522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":367153635,"num_objects":89638,"num_object_clones":0,"num_object_copies":179276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3757,"num_objects_unfound":0,"num_objects_dirty":89638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45437,"num_write_kb":181748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":631,"num_bytes_recovered":2584576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":735207424,"data_stored":735117099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89719,"ondisk_log_size":89719,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503832,"kb_used_data":319432,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93868008,"statfs":{"total":96636764160,"available":96120840192,"internally_reserved":0,"allocated":327098368,"data_stored":326853278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,3,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518392,"kb_used_data":342376,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93853448,"statfs":{"total":96636764160,"available":96105930752,"internally_reserved":0,"allocated":350593024,"data_stored":350343857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,0,4,9,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150088,"kb_used_data":55928,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94221752,"statfs":{"total":96636764160,"available":96483074048,"internally_reserved":0,"allocated":57270272,"data_stored":56941619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351178752,"data_stored":351175139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57171968,"data_stored":57088869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326856704,"data_stored":326853091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:53.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:13:53.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5587", 'reported_seq': 5655, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:51.375454+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:13:51.375454+0000', 'last_peered': '2024-09-05T13:13:51.375454+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:13:51.375454+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 5587, 'log_dups_size': 0, 'ondisk_log_size': 5587, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22884352, 'num_objects': 5587, 'num_object_clones': 0, 'num_object_copies': 11174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5587, 'num_write_kb': 22348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1166}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:53.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:13:53.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5532", 'reported_seq': 5654, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:47.179807+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:13:47.179807+0000', 'last_peered': '2024-09-05T13:13:47.179807+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:13:47.179807+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 5532, 'log_dups_size': 0, 'ondisk_log_size': 5532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22642688, 'num_objects': 5528, 'num_object_clones': 0, 'num_object_copies': 11056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3343, 'num_write_kb': 13372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 1036}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:53.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:13:53.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5669", 'reported_seq': 5799, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:51.411192+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:13:51.411192+0000', 'last_peered': '2024-09-05T13:13:51.411192+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:13:51.411192+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 5669, 'log_dups_size': 0, 'ondisk_log_size': 5669, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23199744, 'num_objects': 5664, 'num_object_clones': 0, 'num_object_copies': 11328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2124, 'num_write_kb': 8496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1213}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:53.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:13:53.465 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5605", 'reported_seq': 5739, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:51.360190+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:13:51.360190+0000', 'last_peered': '2024-09-05T13:13:51.360190+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:13:51.360190+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 5605, 'log_dups_size': 0, 'ondisk_log_size': 5605, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22933504, 'num_objects': 5599, 'num_object_clones': 0, 'num_object_copies': 11198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1924, 'num_write_kb': 7696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1163}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5518", 'reported_seq': 5647, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:47.270480+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:13:47.270480+0000', 'last_peered': '2024-09-05T13:13:47.270480+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:13:47.270480+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 5518, 'log_dups_size': 0, 'ondisk_log_size': 5518, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22573056, 'num_objects': 5511, 'num_object_clones': 0, 'num_object_copies': 11022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5511, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3172, 'num_write_kb': 12688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 1060}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5667", 'reported_seq': 5799, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:51.407653+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:13:51.407653+0000', 'last_peered': '2024-09-05T13:13:51.407653+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:13:51.407653+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 5667, 'log_dups_size': 0, 'ondisk_log_size': 5667, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23179264, 'num_objects': 5659, 'num_object_clones': 0, 'num_object_copies': 11318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2034, 'num_write_kb': 8136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1205}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5460", 'reported_seq': 6826, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:47.261024+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:13:47.261024+0000', 'last_peered': '2024-09-05T13:13:47.261024+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:13:47.261024+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 5460, 'log_dups_size': 0, 'ondisk_log_size': 5460, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22343680, 'num_objects': 5455, 'num_object_clones': 0, 'num_object_copies': 10910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3757, 'num_objects_unfound': 0, 'num_objects_dirty': 5455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3055, 'num_write_kb': 12220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 628, 'num_bytes_recovered': 2572288, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3757}, {'shards': '2', 'objects': 1696}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5464", 'reported_seq': 5581, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:47.255611+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:13:47.255611+0000', 'last_peered': '2024-09-05T13:13:47.255611+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:13:47.255611+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 5464, 'log_dups_size': 0, 'ondisk_log_size': 5464, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22355968, 'num_objects': 5458, 'num_object_clones': 0, 'num_object_copies': 10916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2973, 'num_write_kb': 11892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 1071}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5619", 'reported_seq': 5754, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:51.311965+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:13:51.311965+0000', 'last_peered': '2024-09-05T13:13:51.311965+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:13:51.311965+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 5619, 'log_dups_size': 0, 'ondisk_log_size': 5619, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23007232, 'num_objects': 5617, 'num_object_clones': 0, 'num_object_copies': 11234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1770, 'num_write_kb': 7080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1198}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:53.466 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:13:53.467 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5680", 'reported_seq': 5862, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:51.403751+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:13:51.403751+0000', 'last_peered': '2024-09-05T13:13:51.403751+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:13:51.403751+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 5680, 'log_dups_size': 0, 'ondisk_log_size': 5680, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23232512, 'num_objects': 5672, 'num_object_clones': 0, 'num_object_copies': 11344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1579, 'num_write_kb': 6316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1228}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:53.467 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:13:53.467 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5730", 'reported_seq': 5863, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:51.359772+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:13:51.359772+0000', 'last_peered': '2024-09-05T13:13:51.359772+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:13:51.359772+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 5730, 'log_dups_size': 0, 'ondisk_log_size': 5730, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23453696, 'num_objects': 5726, 'num_object_clones': 0, 'num_object_copies': 11452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1860, 'num_write_kb': 7440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1160}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:53.467 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:13:53.467 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5772", 'reported_seq': 5906, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:51.372404+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:13:51.372404+0000', 'last_peered': '2024-09-05T13:13:51.372404+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:13:51.372404+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 5772, 'log_dups_size': 0, 'ondisk_log_size': 5772, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23593443, 'num_objects': 5761, 'num_object_clones': 0, 'num_object_copies': 11522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5761, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2440, 'num_write_kb': 9760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1196}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:53.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:13:54.154 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:54.155 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/181382 objects degraded (27.074%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":161,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":90691,"data_bytes":371458550,"bytes_used":1193545728,"bytes_avail":288716746752,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":181382,"degraded_ratio":0.27074351368934074,"misplaced_objects":3640,"misplaced_total":181382,"misplaced_ratio":0.020068143476199403,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":120452,"recovering_keys_per_sec":0,"num_objects_recovered":353,"num_bytes_recovered":1445888,"num_keys_recovered":0,"write_bytes_sec":2327148,"read_op_per_sec":0,"write_op_per_sec":568},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:13:54.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:13:54.180 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:54.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 81 20 92018 91998 4.4356 2.01172 0.0398495 0.0176031 2024-09-05T13:13:54.756 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:54.757 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:54.783 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-05T13:13:53.532482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":371458550,"num_objects":90691,"num_object_clones":0,"num_object_copies":181382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3640,"num_objects_unfound":0,"num_objects_dirty":90691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46490,"num_write_kb":185954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":749,"num_bytes_recovered":3063827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90773,"ondisk_log_size":90773,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1165572,"kb_used_data":723476,"kb_used_omap":12,"kb_used_meta":442035,"kb_avail":281949948,"statfs":{"total":289910292480,"available":288716746752,"internally_reserved":0,"allocated":740839424,"data_stored":740016514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":452643846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,1,10,16,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27934720,"num_objects":6820,"num_object_clones":0,"num_object_copies":13640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-353,"num_objects_unfound":0,"num_objects_dirty":6820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6820,"num_write_kb":27280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":1445888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003840"},"pg_stats":[{"pgid":"2.f","version":"43'5587","reported_seq":5655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.375454+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:51.375454+0000","last_peered":"2024-09-05T13:13:51.375454+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:51.375454+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5587,"log_dups_size":0,"ondisk_log_size":5587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22884352,"num_objects":5587,"num_object_clones":0,"num_object_copies":11174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5587,"num_write_kb":22348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1166},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5679","reported_seq":5778,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:52.242135+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:52.242135+0000","last_peered":"2024-09-05T13:13:52.242135+0000","last_clean":"2024-09-05T13:13:52.242135+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:52.242135+0000","last_undegraded":"2024-09-05T13:13:52.242135+0000","last_fullsized":"2024-09-05T13:13:52.242135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5679,"log_dups_size":0,"ondisk_log_size":5679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23236608,"num_objects":5673,"num_object_clones":0,"num_object_copies":11346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3689,"num_write_kb":14756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5702","reported_seq":5824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:52.229309+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:52.229309+0000","last_peered":"2024-09-05T13:13:52.229309+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:52.229309+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23339008,"num_objects":5698,"num_object_clones":0,"num_object_copies":11396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3513,"num_write_kb":14052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1206}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5730","reported_seq":5794,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:52.289143+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:52.289143+0000","last_peered":"2024-09-05T13:13:52.289143+0000","last_clean":"2024-09-05T13:13:52.289143+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:52.289143+0000","last_undegraded":"2024-09-05T13:13:52.289143+0000","last_fullsized":"2024-09-05T13:13:52.289143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23470080,"num_objects":5730,"num_object_clones":0,"num_object_copies":11460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5671","reported_seq":5800,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:51.410863+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:51.410863+0000","last_peered":"2024-09-05T13:13:51.410863+0000","last_clean":"2024-09-05T13:13:51.410863+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:51.410863+0000","last_undegraded":"2024-09-05T13:13:51.410863+0000","last_fullsized":"2024-09-05T13:13:51.410863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5671,"log_dups_size":0,"ondisk_log_size":5671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2211,"num_write_kb":8844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5669","reported_seq":5799,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.411192+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:51.411192+0000","last_peered":"2024-09-05T13:13:51.411192+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:51.411192+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5669,"log_dups_size":0,"ondisk_log_size":5669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2124,"num_write_kb":8496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5605","reported_seq":5739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.360190+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:51.360190+0000","last_peered":"2024-09-05T13:13:51.360190+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:51.360190+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5605,"log_dups_size":0,"ondisk_log_size":5605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933504,"num_objects":5599,"num_object_clones":0,"num_object_copies":11198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1924,"num_write_kb":7696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1163},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5725","reported_seq":5851,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:51.393687+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:51.393687+0000","last_peered":"2024-09-05T13:13:51.393687+0000","last_clean":"2024-09-05T13:13:51.393687+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:51.393687+0000","last_undegraded":"2024-09-05T13:13:51.393687+0000","last_fullsized":"2024-09-05T13:13:51.393687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5725,"log_dups_size":0,"ondisk_log_size":5725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23441408,"num_objects":5723,"num_object_clones":0,"num_object_copies":11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2335,"num_write_kb":9340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5697","reported_seq":5826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:52.210332+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:52.210332+0000","last_peered":"2024-09-05T13:13:52.210332+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:52.210332+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5697,"log_dups_size":0,"ondisk_log_size":5697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3351,"num_write_kb":13404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5667","reported_seq":5799,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.407653+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:51.407653+0000","last_peered":"2024-09-05T13:13:51.407653+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:51.407653+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5667,"log_dups_size":0,"ondisk_log_size":5667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23179264,"num_objects":5659,"num_object_clones":0,"num_object_copies":11318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2034,"num_write_kb":8136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1205},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5630","reported_seq":7230,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:52.265845+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:52.265845+0000","last_peered":"2024-09-05T13:13:52.265845+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:52.265845+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5630,"log_dups_size":0,"ondisk_log_size":5630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23040000,"num_objects":5625,"num_object_clones":0,"num_object_copies":11250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3640,"num_objects_unfound":0,"num_objects_dirty":5625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3225,"num_write_kb":12900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":745,"num_bytes_recovered":3051520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3640},{"shards":"2","objects":1983}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5607","reported_seq":5724,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:52.265895+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:52.265895+0000","last_peered":"2024-09-05T13:13:52.265895+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:52.265895+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5607,"log_dups_size":0,"ondisk_log_size":5607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22941696,"num_objects":5601,"num_object_clones":0,"num_object_copies":11202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3116,"num_write_kb":12464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1214}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5619","reported_seq":5754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.311965+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:51.311965+0000","last_peered":"2024-09-05T13:13:51.311965+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:51.311965+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5619,"log_dups_size":0,"ondisk_log_size":5619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1770,"num_write_kb":7080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1198},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5680","reported_seq":5862,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.403751+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:51.403751+0000","last_peered":"2024-09-05T13:13:51.403751+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:51.403751+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5680,"log_dups_size":0,"ondisk_log_size":5680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1579,"num_write_kb":6316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1228},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5730","reported_seq":5863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.359772+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:51.359772+0000","last_peered":"2024-09-05T13:13:51.359772+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:51.359772+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5730,"log_dups_size":0,"ondisk_log_size":5730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1860,"num_write_kb":7440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1160},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5772","reported_seq":5906,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:51.372404+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:51.372404+0000","last_peered":"2024-09-05T13:13:51.372404+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:51.372404+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5772,"log_dups_size":0,"ondisk_log_size":5772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23593443,"num_objects":5761,"num_object_clones":0,"num_object_copies":11522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1196},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":371458531,"num_objects":90689,"num_object_clones":0,"num_object_copies":181378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3640,"num_objects_unfound":0,"num_objects_dirty":90689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46488,"num_write_kb":185952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":748,"num_bytes_recovered":3063808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":741081088,"data_stored":740990763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90770,"ondisk_log_size":90770,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497092,"kb_used_data":325172,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93874748,"statfs":{"total":96636764160,"available":96127741952,"internally_reserved":0,"allocated":332976128,"data_stored":332731038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":176025261},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518392,"kb_used_data":342376,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93853448,"statfs":{"total":96636764160,"available":96105930752,"internally_reserved":0,"allocated":350593024,"data_stored":350343857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,0,4,9,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150088,"kb_used_data":55928,"kb_used_omap":4,"kb_used_meta":94139,"kb_avail":94221752,"statfs":{"total":96636764160,"available":96483074048,"internally_reserved":0,"allocated":57270272,"data_stored":56941619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96399023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351178752,"data_stored":351175139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57171968,"data_stored":57088869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332730368,"data_stored":332726755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:55.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 82 20 92577 92557 4.40814 2.18359 0.066964 0.0177142 2024-09-05T13:13:56.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 83 20 93167 93147 4.3828 2.30469 0.0109429 0.0178175 2024-09-05T13:13:57.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 84 20 93754 93734 4.35792 2.29297 0.0269084 0.0179187 2024-09-05T13:13:57.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:58.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 85 20 94305 94285 4.33198 2.15234 0.0139696 0.0180252 2024-09-05T13:13:58.333 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:58.333 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:58.361 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-05T13:13:57.533623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":383042038,"num_objects":93519,"num_object_clones":0,"num_object_copies":187038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":93519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49318,"num_write_kb":197266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":859,"num_bytes_recovered":3514387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93601,"ondisk_log_size":93601,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1210492,"kb_used_data":746188,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281905028,"statfs":{"total":289910292480,"available":288670748672,"internally_reserved":0,"allocated":764096512,"data_stored":763273602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":475384838},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,9,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35033088,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-340,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8553,"num_write_kb":34212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003665"},"pg_stats":[{"pgid":"2.f","version":"43'5778","reported_seq":5847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.389338+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:56.389338+0000","last_peered":"2024-09-05T13:13:56.389338+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:56.389338+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5778,"num_write_kb":23112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1357},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5869","reported_seq":5968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:57.250843+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:57.250843+0000","last_peered":"2024-09-05T13:13:57.250843+0000","last_clean":"2024-09-05T13:13:57.250843+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:57.250843+0000","last_undegraded":"2024-09-05T13:13:57.250843+0000","last_fullsized":"2024-09-05T13:13:57.250843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3879,"num_write_kb":15516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5899","reported_seq":6021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.336629+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:57.336629+0000","last_peered":"2024-09-05T13:13:57.336629+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:57.336629+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5899,"log_dups_size":0,"ondisk_log_size":5899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24145920,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3710,"num_write_kb":14840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5896","reported_seq":5960,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:57.305020+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:57.305020+0000","last_peered":"2024-09-05T13:13:57.305020+0000","last_clean":"2024-09-05T13:13:57.305020+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:57.305020+0000","last_undegraded":"2024-09-05T13:13:57.305020+0000","last_fullsized":"2024-09-05T13:13:57.305020+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5896,"log_dups_size":0,"ondisk_log_size":5896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150016,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":23584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5867","reported_seq":5996,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:56.461806+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:56.461806+0000","last_peered":"2024-09-05T13:13:56.461806+0000","last_clean":"2024-09-05T13:13:56.461806+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:56.461806+0000","last_undegraded":"2024-09-05T13:13:56.461806+0000","last_fullsized":"2024-09-05T13:13:56.461806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5867,"log_dups_size":0,"ondisk_log_size":5867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24002560,"num_objects":5860,"num_object_clones":0,"num_object_copies":11720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2407,"num_write_kb":9628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5832","reported_seq":5965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.396821+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:56.396821+0000","last_peered":"2024-09-05T13:13:56.396821+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:56.396821+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5832,"log_dups_size":0,"ondisk_log_size":5832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867392,"num_objects":5827,"num_object_clones":0,"num_object_copies":11654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2287,"num_write_kb":9148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1376},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5774","reported_seq":5908,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.406102+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:56.406102+0000","last_peered":"2024-09-05T13:13:56.406102+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:56.406102+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23625728,"num_objects":5768,"num_object_clones":0,"num_object_copies":11536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2093,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5910","reported_seq":6036,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:56.453429+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:56.453429+0000","last_peered":"2024-09-05T13:13:56.453429+0000","last_clean":"2024-09-05T13:13:56.453429+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:56.453429+0000","last_undegraded":"2024-09-05T13:13:56.453429+0000","last_fullsized":"2024-09-05T13:13:56.453429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5910,"log_dups_size":0,"ondisk_log_size":5910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24199168,"num_objects":5908,"num_object_clones":0,"num_object_copies":11816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2520,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5875","reported_seq":6004,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.245142+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:57.245142+0000","last_peered":"2024-09-05T13:13:57.245142+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:57.245142+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5875,"log_dups_size":0,"ondisk_log_size":5875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24035328,"num_objects":5868,"num_object_clones":0,"num_object_copies":11736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3529,"num_write_kb":14116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1417}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5819","reported_seq":5951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.358231+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:56.358231+0000","last_peered":"2024-09-05T13:13:56.358231+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:56.358231+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5819,"log_dups_size":0,"ondisk_log_size":5819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2186,"num_write_kb":8744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1357},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5804","reported_seq":7624,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.325593+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:57.325593+0000","last_peered":"2024-09-05T13:13:57.325593+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:57.325593+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5804,"log_dups_size":0,"ondisk_log_size":5804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23752704,"num_objects":5799,"num_object_clones":0,"num_object_copies":11598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":5799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3399,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":855,"num_bytes_recovered":3502080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3530},{"shards":"2","objects":2267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5783","reported_seq":5897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.320904+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:57.320904+0000","last_peered":"2024-09-05T13:13:57.320904+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:57.320904+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5783,"log_dups_size":0,"ondisk_log_size":5783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23662592,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3292,"num_write_kb":13168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1390}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5792","reported_seq":5926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.430766+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:56.430766+0000","last_peered":"2024-09-05T13:13:56.430766+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:56.430766+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5792,"log_dups_size":0,"ondisk_log_size":5792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23715840,"num_objects":5790,"num_object_clones":0,"num_object_copies":11580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1943,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1371},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5859","reported_seq":6041,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.441338+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:56.441338+0000","last_peered":"2024-09-05T13:13:56.441338+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:56.441338+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23965696,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1758,"num_write_kb":7032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1407},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5899","reported_seq":6032,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.441188+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:56.441188+0000","last_peered":"2024-09-05T13:13:56.441188+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:56.441188+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5899,"log_dups_size":0,"ondisk_log_size":5899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24145920,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2029,"num_write_kb":8116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1329},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5942","reported_seq":6076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.463502+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:56.463502+0000","last_peered":"2024-09-05T13:13:56.463502+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:56.463502+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5942,"log_dups_size":0,"ondisk_log_size":5942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289763,"num_objects":5931,"num_object_clones":0,"num_object_copies":11862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2610,"num_write_kb":10440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1366},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":383042019,"num_objects":93517,"num_object_clones":0,"num_object_copies":187034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":93517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49316,"num_write_kb":197264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":858,"num_bytes_recovered":3514368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":764678144,"data_stored":764587819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93598,"ondisk_log_size":93598,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511108,"kb_used_data":330868,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93860732,"statfs":{"total":96636764160,"available":96113389568,"internally_reserved":0,"allocated":338808832,"data_stored":338563742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535056,"kb_used_data":350720,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93836784,"statfs":{"total":96636764160,"available":96088866816,"internally_reserved":0,"allocated":359137280,"data_stored":358888113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164328,"kb_used_data":64600,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94207512,"statfs":{"total":96636764160,"available":96468492288,"internally_reserved":0,"allocated":66150400,"data_stored":65821747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":102100655},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":359792640,"data_stored":359789027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66215936,"data_stored":66132837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338669568,"data_stored":338665955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:58.362 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:13:58.942 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:58.942 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:13:58.964 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-05T13:13:57.533623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":383042038,"num_objects":93519,"num_object_clones":0,"num_object_copies":187038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":93519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49318,"num_write_kb":197266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":859,"num_bytes_recovered":3514387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93601,"ondisk_log_size":93601,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1210492,"kb_used_data":746188,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281905028,"statfs":{"total":289910292480,"available":288670748672,"internally_reserved":0,"allocated":764096512,"data_stored":763273602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":475384838},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,9,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35033088,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-340,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8553,"num_write_kb":34212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003665"},"pg_stats":[{"pgid":"2.f","version":"43'5778","reported_seq":5847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.389338+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:56.389338+0000","last_peered":"2024-09-05T13:13:56.389338+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:56.389338+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5778,"num_write_kb":23112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1357},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5869","reported_seq":5968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:57.250843+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:57.250843+0000","last_peered":"2024-09-05T13:13:57.250843+0000","last_clean":"2024-09-05T13:13:57.250843+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:57.250843+0000","last_undegraded":"2024-09-05T13:13:57.250843+0000","last_fullsized":"2024-09-05T13:13:57.250843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3879,"num_write_kb":15516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5899","reported_seq":6021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.336629+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:57.336629+0000","last_peered":"2024-09-05T13:13:57.336629+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:57.336629+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5899,"log_dups_size":0,"ondisk_log_size":5899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24145920,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3710,"num_write_kb":14840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5896","reported_seq":5960,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:57.305020+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:57.305020+0000","last_peered":"2024-09-05T13:13:57.305020+0000","last_clean":"2024-09-05T13:13:57.305020+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:57.305020+0000","last_undegraded":"2024-09-05T13:13:57.305020+0000","last_fullsized":"2024-09-05T13:13:57.305020+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5896,"log_dups_size":0,"ondisk_log_size":5896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150016,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":23584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5867","reported_seq":5996,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:56.461806+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:56.461806+0000","last_peered":"2024-09-05T13:13:56.461806+0000","last_clean":"2024-09-05T13:13:56.461806+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:56.461806+0000","last_undegraded":"2024-09-05T13:13:56.461806+0000","last_fullsized":"2024-09-05T13:13:56.461806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5867,"log_dups_size":0,"ondisk_log_size":5867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24002560,"num_objects":5860,"num_object_clones":0,"num_object_copies":11720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2407,"num_write_kb":9628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5832","reported_seq":5965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.396821+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:56.396821+0000","last_peered":"2024-09-05T13:13:56.396821+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:56.396821+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5832,"log_dups_size":0,"ondisk_log_size":5832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867392,"num_objects":5827,"num_object_clones":0,"num_object_copies":11654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2287,"num_write_kb":9148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1376},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5774","reported_seq":5908,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.406102+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:56.406102+0000","last_peered":"2024-09-05T13:13:56.406102+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:56.406102+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23625728,"num_objects":5768,"num_object_clones":0,"num_object_copies":11536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2093,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5910","reported_seq":6036,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:56.453429+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:56.453429+0000","last_peered":"2024-09-05T13:13:56.453429+0000","last_clean":"2024-09-05T13:13:56.453429+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:56.453429+0000","last_undegraded":"2024-09-05T13:13:56.453429+0000","last_fullsized":"2024-09-05T13:13:56.453429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5910,"log_dups_size":0,"ondisk_log_size":5910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24199168,"num_objects":5908,"num_object_clones":0,"num_object_copies":11816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2520,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5875","reported_seq":6004,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.245142+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:57.245142+0000","last_peered":"2024-09-05T13:13:57.245142+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:57.245142+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5875,"log_dups_size":0,"ondisk_log_size":5875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24035328,"num_objects":5868,"num_object_clones":0,"num_object_copies":11736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3529,"num_write_kb":14116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1417}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5819","reported_seq":5951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.358231+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:56.358231+0000","last_peered":"2024-09-05T13:13:56.358231+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:56.358231+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5819,"log_dups_size":0,"ondisk_log_size":5819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2186,"num_write_kb":8744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1357},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5804","reported_seq":7624,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.325593+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:57.325593+0000","last_peered":"2024-09-05T13:13:57.325593+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:57.325593+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5804,"log_dups_size":0,"ondisk_log_size":5804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23752704,"num_objects":5799,"num_object_clones":0,"num_object_copies":11598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":5799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3399,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":855,"num_bytes_recovered":3502080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3530},{"shards":"2","objects":2267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5783","reported_seq":5897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.320904+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:57.320904+0000","last_peered":"2024-09-05T13:13:57.320904+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:57.320904+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5783,"log_dups_size":0,"ondisk_log_size":5783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23662592,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3292,"num_write_kb":13168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1390}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5792","reported_seq":5926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.430766+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:56.430766+0000","last_peered":"2024-09-05T13:13:56.430766+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:56.430766+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5792,"log_dups_size":0,"ondisk_log_size":5792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23715840,"num_objects":5790,"num_object_clones":0,"num_object_copies":11580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1943,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1371},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5859","reported_seq":6041,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.441338+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:56.441338+0000","last_peered":"2024-09-05T13:13:56.441338+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:56.441338+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23965696,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1758,"num_write_kb":7032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1407},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5899","reported_seq":6032,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.441188+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:56.441188+0000","last_peered":"2024-09-05T13:13:56.441188+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:56.441188+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5899,"log_dups_size":0,"ondisk_log_size":5899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24145920,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2029,"num_write_kb":8116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1329},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5942","reported_seq":6076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.463502+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:56.463502+0000","last_peered":"2024-09-05T13:13:56.463502+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:56.463502+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5942,"log_dups_size":0,"ondisk_log_size":5942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289763,"num_objects":5931,"num_object_clones":0,"num_object_copies":11862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2610,"num_write_kb":10440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1366},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":383042019,"num_objects":93517,"num_object_clones":0,"num_object_copies":187034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":93517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49316,"num_write_kb":197264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":858,"num_bytes_recovered":3514368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":764678144,"data_stored":764587819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93598,"ondisk_log_size":93598,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511108,"kb_used_data":330868,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93860732,"statfs":{"total":96636764160,"available":96113389568,"internally_reserved":0,"allocated":338808832,"data_stored":338563742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535056,"kb_used_data":350720,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93836784,"statfs":{"total":96636764160,"available":96088866816,"internally_reserved":0,"allocated":359137280,"data_stored":358888113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164328,"kb_used_data":64600,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94207512,"statfs":{"total":96636764160,"available":96468492288,"internally_reserved":0,"allocated":66150400,"data_stored":65821747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":102100655},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":359792640,"data_stored":359789027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66215936,"data_stored":66132837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338669568,"data_stored":338665955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:13:58.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:13:58.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5778", 'reported_seq': 5847, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:56.389338+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:13:56.389338+0000', 'last_peered': '2024-09-05T13:13:56.389338+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:13:56.389338+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 5778, 'log_dups_size': 0, 'ondisk_log_size': 5778, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23666688, 'num_objects': 5778, 'num_object_clones': 0, 'num_object_copies': 11556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5778, 'num_write_kb': 23112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1357}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:58.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:13:58.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5899", 'reported_seq': 6021, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:57.336629+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:13:57.336629+0000', 'last_peered': '2024-09-05T13:13:57.336629+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:13:57.336629+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 5899, 'log_dups_size': 0, 'ondisk_log_size': 5899, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24145920, 'num_objects': 5895, 'num_object_clones': 0, 'num_object_copies': 11790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3710, 'num_write_kb': 14840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 1403}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:58.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:13:58.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5832", 'reported_seq': 5965, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:56.396821+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:13:56.396821+0000', 'last_peered': '2024-09-05T13:13:56.396821+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:13:56.396821+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 5832, 'log_dups_size': 0, 'ondisk_log_size': 5832, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23867392, 'num_objects': 5827, 'num_object_clones': 0, 'num_object_copies': 11654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2287, 'num_write_kb': 9148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1376}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:58.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5774", 'reported_seq': 5908, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:56.406102+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:13:56.406102+0000', 'last_peered': '2024-09-05T13:13:56.406102+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:13:56.406102+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 5774, 'log_dups_size': 0, 'ondisk_log_size': 5774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23625728, 'num_objects': 5768, 'num_object_clones': 0, 'num_object_copies': 11536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2093, 'num_write_kb': 8372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1332}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5875", 'reported_seq': 6004, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:57.245142+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:13:57.245142+0000', 'last_peered': '2024-09-05T13:13:57.245142+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:13:57.245142+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 5875, 'log_dups_size': 0, 'ondisk_log_size': 5875, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24035328, 'num_objects': 5868, 'num_object_clones': 0, 'num_object_copies': 11736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3529, 'num_write_kb': 14116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 1417}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5819", 'reported_seq': 5951, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:56.358231+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:13:56.358231+0000', 'last_peered': '2024-09-05T13:13:56.358231+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:13:56.358231+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 5819, 'log_dups_size': 0, 'ondisk_log_size': 5819, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23801856, 'num_objects': 5811, 'num_object_clones': 0, 'num_object_copies': 11622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2186, 'num_write_kb': 8744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1357}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5804", 'reported_seq': 7624, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:57.325593+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:13:57.325593+0000', 'last_peered': '2024-09-05T13:13:57.325593+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:13:57.325593+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 5804, 'log_dups_size': 0, 'ondisk_log_size': 5804, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23752704, 'num_objects': 5799, 'num_object_clones': 0, 'num_object_copies': 11598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3530, 'num_objects_unfound': 0, 'num_objects_dirty': 5799, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3399, 'num_write_kb': 13596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 855, 'num_bytes_recovered': 3502080, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3530}, {'shards': '2', 'objects': 2267}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5783", 'reported_seq': 5897, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:57.320904+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:13:57.320904+0000', 'last_peered': '2024-09-05T13:13:57.320904+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:13:57.320904+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 5783, 'log_dups_size': 0, 'ondisk_log_size': 5783, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23662592, 'num_objects': 5777, 'num_object_clones': 0, 'num_object_copies': 11554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3292, 'num_write_kb': 13168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 1390}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:13:58.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:13:58.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5792", 'reported_seq': 5926, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:56.430766+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:13:56.430766+0000', 'last_peered': '2024-09-05T13:13:56.430766+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:13:56.430766+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 5792, 'log_dups_size': 0, 'ondisk_log_size': 5792, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23715840, 'num_objects': 5790, 'num_object_clones': 0, 'num_object_copies': 11580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5790, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1943, 'num_write_kb': 7772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1371}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:58.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:13:58.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5859", 'reported_seq': 6041, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:56.441338+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:13:56.441338+0000', 'last_peered': '2024-09-05T13:13:56.441338+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:13:56.441338+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 5859, 'log_dups_size': 0, 'ondisk_log_size': 5859, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23965696, 'num_objects': 5851, 'num_object_clones': 0, 'num_object_copies': 11702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1758, 'num_write_kb': 7032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1407}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:58.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:13:58.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5899", 'reported_seq': 6032, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:56.441188+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:13:56.441188+0000', 'last_peered': '2024-09-05T13:13:56.441188+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:13:56.441188+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 5899, 'log_dups_size': 0, 'ondisk_log_size': 5899, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24145920, 'num_objects': 5895, 'num_object_clones': 0, 'num_object_copies': 11790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2029, 'num_write_kb': 8116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1329}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:58.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:13:58.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5942", 'reported_seq': 6076, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:13:56.463502+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:13:56.463502+0000', 'last_peered': '2024-09-05T13:13:56.463502+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:13:56.463502+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 5942, 'log_dups_size': 0, 'ondisk_log_size': 5942, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24289763, 'num_objects': 5931, 'num_object_clones': 0, 'num_object_copies': 11862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2610, 'num_write_kb': 10440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1366}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:13:58.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:13:59.204 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 86 20 94819 94799 4.30495 2.00781 0.0418065 0.0181396 2024-09-05T13:13:59.659 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:13:59.659 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/187038 objects degraded (26.256%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":166,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":93519,"data_bytes":383042038,"bytes_used":1239543808,"bytes_avail":288670748672,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":187038,"degraded_ratio":0.26255627198751058,"misplaced_objects":3530,"misplaced_total":187038,"misplaced_ratio":0.018873170157935819,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":116017,"recovering_keys_per_sec":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"write_bytes_sec":2918532,"read_op_per_sec":0,"write_op_per_sec":712},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:13:59.688 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:13:59.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:00.205 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 87 20 95318 95298 4.27782 1.94922 0.016126 0.0182529 2024-09-05T13:14:00.265 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:00.266 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:00.296 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-05T13:13:59.534010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":383042038,"num_objects":93519,"num_object_clones":0,"num_object_copies":187038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":93519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49318,"num_write_kb":197266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":859,"num_bytes_recovered":3514387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93601,"ondisk_log_size":93601,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1227528,"kb_used_data":754904,"kb_used_omap":12,"kb_used_meta":472563,"kb_avail":281887992,"statfs":{"total":289910292480,"available":288653303808,"internally_reserved":0,"allocated":773021696,"data_stored":772198786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":483904518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,5,7,14,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23273472,"num_objects":5682,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-227,"num_objects_unfound":0,"num_objects_dirty":5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":929792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003635"},"pg_stats":[{"pgid":"2.f","version":"43'5778","reported_seq":5847,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.389338+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:13:56.389338+0000","last_peered":"2024-09-05T13:13:56.389338+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:13:56.389338+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5778,"num_write_kb":23112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1357},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5869","reported_seq":5968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:57.250843+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:13:57.250843+0000","last_peered":"2024-09-05T13:13:57.250843+0000","last_clean":"2024-09-05T13:13:57.250843+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:13:57.250843+0000","last_undegraded":"2024-09-05T13:13:57.250843+0000","last_fullsized":"2024-09-05T13:13:57.250843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3879,"num_write_kb":15516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5899","reported_seq":6021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.336629+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:13:57.336629+0000","last_peered":"2024-09-05T13:13:57.336629+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:13:57.336629+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":5899,"log_dups_size":0,"ondisk_log_size":5899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24145920,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3710,"num_write_kb":14840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1403}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5896","reported_seq":5960,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:57.305020+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:13:57.305020+0000","last_peered":"2024-09-05T13:13:57.305020+0000","last_clean":"2024-09-05T13:13:57.305020+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:13:57.305020+0000","last_undegraded":"2024-09-05T13:13:57.305020+0000","last_fullsized":"2024-09-05T13:13:57.305020+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5896,"log_dups_size":0,"ondisk_log_size":5896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150016,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":23584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5867","reported_seq":5996,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:56.461806+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:13:56.461806+0000","last_peered":"2024-09-05T13:13:56.461806+0000","last_clean":"2024-09-05T13:13:56.461806+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:13:56.461806+0000","last_undegraded":"2024-09-05T13:13:56.461806+0000","last_fullsized":"2024-09-05T13:13:56.461806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":5867,"log_dups_size":0,"ondisk_log_size":5867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24002560,"num_objects":5860,"num_object_clones":0,"num_object_copies":11720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2407,"num_write_kb":9628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5832","reported_seq":5965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.396821+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:13:56.396821+0000","last_peered":"2024-09-05T13:13:56.396821+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:13:56.396821+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":5832,"log_dups_size":0,"ondisk_log_size":5832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867392,"num_objects":5827,"num_object_clones":0,"num_object_copies":11654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2287,"num_write_kb":9148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1376},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5774","reported_seq":5908,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.406102+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:13:56.406102+0000","last_peered":"2024-09-05T13:13:56.406102+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:13:56.406102+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5774,"log_dups_size":0,"ondisk_log_size":5774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23625728,"num_objects":5768,"num_object_clones":0,"num_object_copies":11536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2093,"num_write_kb":8372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5910","reported_seq":6036,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:56.453429+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:13:56.453429+0000","last_peered":"2024-09-05T13:13:56.453429+0000","last_clean":"2024-09-05T13:13:56.453429+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:13:56.453429+0000","last_undegraded":"2024-09-05T13:13:56.453429+0000","last_fullsized":"2024-09-05T13:13:56.453429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":5910,"log_dups_size":0,"ondisk_log_size":5910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24199168,"num_objects":5908,"num_object_clones":0,"num_object_copies":11816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2520,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5875","reported_seq":6004,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.245142+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:13:57.245142+0000","last_peered":"2024-09-05T13:13:57.245142+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:13:57.245142+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":5875,"log_dups_size":0,"ondisk_log_size":5875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24035328,"num_objects":5868,"num_object_clones":0,"num_object_copies":11736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3529,"num_write_kb":14116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1417}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5819","reported_seq":5951,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.358231+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:13:56.358231+0000","last_peered":"2024-09-05T13:13:56.358231+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:13:56.358231+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":5819,"log_dups_size":0,"ondisk_log_size":5819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2186,"num_write_kb":8744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1357},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5804","reported_seq":7624,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.325593+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:13:57.325593+0000","last_peered":"2024-09-05T13:13:57.325593+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:13:57.325593+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5804,"log_dups_size":0,"ondisk_log_size":5804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23752704,"num_objects":5799,"num_object_clones":0,"num_object_copies":11598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":5799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3399,"num_write_kb":13596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":855,"num_bytes_recovered":3502080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3530},{"shards":"2","objects":2267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5783","reported_seq":5897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:57.320904+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:13:57.320904+0000","last_peered":"2024-09-05T13:13:57.320904+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:13:57.320904+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5783,"log_dups_size":0,"ondisk_log_size":5783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23662592,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3292,"num_write_kb":13168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1390}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5792","reported_seq":5926,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.430766+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:13:56.430766+0000","last_peered":"2024-09-05T13:13:56.430766+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:13:56.430766+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5792,"log_dups_size":0,"ondisk_log_size":5792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23715840,"num_objects":5790,"num_object_clones":0,"num_object_copies":11580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1943,"num_write_kb":7772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1371},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5859","reported_seq":6041,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.441338+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:13:56.441338+0000","last_peered":"2024-09-05T13:13:56.441338+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:13:56.441338+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23965696,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1758,"num_write_kb":7032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1407},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5899","reported_seq":6032,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.441188+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:13:56.441188+0000","last_peered":"2024-09-05T13:13:56.441188+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:13:56.441188+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":5899,"log_dups_size":0,"ondisk_log_size":5899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24145920,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2029,"num_write_kb":8116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1329},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5942","reported_seq":6076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:13:56.463502+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:13:56.463502+0000","last_peered":"2024-09-05T13:13:56.463502+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:13:56.463502+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":5942,"log_dups_size":0,"ondisk_log_size":5942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289763,"num_objects":5931,"num_object_clones":0,"num_object_copies":11862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2610,"num_write_kb":10440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1366},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":383042019,"num_objects":93517,"num_object_clones":0,"num_object_copies":187034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3530,"num_objects_unfound":0,"num_objects_dirty":93517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49316,"num_write_kb":197264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":858,"num_bytes_recovered":3514368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":773693440,"data_stored":773603115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93598,"ondisk_log_size":93598,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511108,"kb_used_data":330868,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93860732,"statfs":{"total":96636764160,"available":96113389568,"internally_reserved":0,"allocated":338808832,"data_stored":338563742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535056,"kb_used_data":350720,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93836784,"statfs":{"total":96636764160,"available":96088866816,"internally_reserved":0,"allocated":359137280,"data_stored":358888113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":181364,"kb_used_data":73316,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94190476,"statfs":{"total":96636764160,"available":96451047424,"internally_reserved":0,"allocated":75075584,"data_stored":74746931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":359792640,"data_stored":359789027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75231232,"data_stored":75148133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338669568,"data_stored":338665955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:01.205 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 88 20 95891 95871 4.25465 2.23828 0.0435188 0.0183545 2024-09-05T13:14:02.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 89 20 96459 96439 4.23173 2.21875 0.0154031 0.0184529 2024-09-05T13:14:03.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 90 20 97052 97032 4.21045 2.31641 0.01521 0.0185446 2024-09-05T13:14:03.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:03.849 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:03.851 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:03.877 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-05T13:14:03.535398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394273270,"num_objects":96261,"num_object_clones":0,"num_object_copies":192522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":96261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52060,"num_write_kb":208234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":975,"num_bytes_recovered":3989523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96343,"ondisk_log_size":96343,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1232880,"kb_used_data":768576,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281882640,"statfs":{"total":289910292480,"available":288647823360,"internally_reserved":0,"allocated":787021824,"data_stored":786198914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":475384838},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,3,5,8,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-343,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6621,"num_write_kb":26484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":343,"num_bytes_recovered":1404928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003650"},"pg_stats":[{"pgid":"2.f","version":"43'5944","reported_seq":6013,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.413584+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:01.413584+0000","last_peered":"2024-09-05T13:14:01.413584+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:01.413584+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5944,"log_dups_size":0,"ondisk_log_size":5944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24346624,"num_objects":5944,"num_object_clones":0,"num_object_copies":11888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":23776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6046","reported_seq":6145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:02.379175+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:02.379175+0000","last_peered":"2024-09-05T13:14:02.379175+0000","last_clean":"2024-09-05T13:14:02.379175+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:02.379175+0000","last_undegraded":"2024-09-05T13:14:02.379175+0000","last_fullsized":"2024-09-05T13:14:02.379175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24739840,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4056,"num_write_kb":16224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6063","reported_seq":6185,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.293505+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:02.293505+0000","last_peered":"2024-09-05T13:14:02.293505+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:02.293505+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6063,"log_dups_size":0,"ondisk_log_size":6063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24817664,"num_objects":6059,"num_object_clones":0,"num_object_copies":12118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3874,"num_write_kb":15496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6078","reported_seq":6139,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:02.388731+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:02.388731+0000","last_peered":"2024-09-05T13:14:02.388731+0000","last_clean":"2024-09-05T13:14:02.388731+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:02.388731+0000","last_undegraded":"2024-09-05T13:14:02.388731+0000","last_fullsized":"2024-09-05T13:14:02.388731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6078,"log_dups_size":0,"ondisk_log_size":6078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24895488,"num_objects":6078,"num_object_clones":0,"num_object_copies":12156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6078,"num_write_kb":24312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6037","reported_seq":6166,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:01.444269+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:01.444269+0000","last_peered":"2024-09-05T13:14:01.444269+0000","last_clean":"2024-09-05T13:14:01.444269+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:01.444269+0000","last_undegraded":"2024-09-05T13:14:01.444269+0000","last_fullsized":"2024-09-05T13:14:01.444269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6037,"log_dups_size":0,"ondisk_log_size":6037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6009","reported_seq":6142,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.462559+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:01.462559+0000","last_peered":"2024-09-05T13:14:01.462559+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:01.462559+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6009,"log_dups_size":0,"ondisk_log_size":6009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2464,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1553},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5962","reported_seq":6093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.511313+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:01.511313+0000","last_peered":"2024-09-05T13:14:01.511313+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:01.511313+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2281,"num_write_kb":9124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1520},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6090","reported_seq":6215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:01.521951+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:01.521951+0000","last_peered":"2024-09-05T13:14:01.521951+0000","last_clean":"2024-09-05T13:14:01.521951+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:01.521951+0000","last_undegraded":"2024-09-05T13:14:01.521951+0000","last_fullsized":"2024-09-05T13:14:01.521951+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6090,"log_dups_size":0,"ondisk_log_size":6090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24936448,"num_objects":6088,"num_object_clones":0,"num_object_copies":12176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2700,"num_write_kb":10800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6057","reported_seq":6186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.370117+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:02.370117+0000","last_peered":"2024-09-05T13:14:02.370117+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:02.370117+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6057,"log_dups_size":0,"ondisk_log_size":6057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3711,"num_write_kb":14844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6011","reported_seq":6142,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.495441+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:01.495441+0000","last_peered":"2024-09-05T13:14:01.495441+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:01.495441+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6011,"log_dups_size":0,"ondisk_log_size":6011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24588288,"num_objects":6003,"num_object_clones":0,"num_object_copies":12006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2378,"num_write_kb":9512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1549},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5965","reported_seq":8018,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.368210+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:02.368210+0000","last_peered":"2024-09-05T13:14:02.368210+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:02.368210+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5965,"log_dups_size":0,"ondisk_log_size":5965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":971,"num_bytes_recovered":3977216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3414},{"shards":"2","objects":2544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5947","reported_seq":6064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.324286+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:02.324286+0000","last_peered":"2024-09-05T13:14:02.324286+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:02.324286+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5947,"log_dups_size":0,"ondisk_log_size":5947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24334336,"num_objects":5941,"num_object_clones":0,"num_object_copies":11882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3456,"num_write_kb":13824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5960","reported_seq":6095,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.439863+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:01.439863+0000","last_peered":"2024-09-05T13:14:01.439863+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:01.439863+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5960,"log_dups_size":0,"ondisk_log_size":5960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24403968,"num_objects":5958,"num_object_clones":0,"num_object_copies":11916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2111,"num_write_kb":8444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1539},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6017","reported_seq":6199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.510984+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:01.510984+0000","last_peered":"2024-09-05T13:14:01.510984+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:01.510984+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24612864,"num_objects":6009,"num_object_clones":0,"num_object_copies":12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1916,"num_write_kb":7664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1565},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6068","reported_seq":6201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.481442+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:01.481442+0000","last_peered":"2024-09-05T13:14:01.481442+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:01.481442+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6068,"log_dups_size":0,"ondisk_log_size":6068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":8792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1498},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6086","reported_seq":6220,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.507209+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:01.507209+0000","last_peered":"2024-09-05T13:14:01.507209+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:01.507209+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6086,"log_dups_size":0,"ondisk_log_size":6086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24879587,"num_objects":6075,"num_object_clones":0,"num_object_copies":12150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2754,"num_write_kb":11016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":394273251,"num_objects":96259,"num_object_clones":0,"num_object_copies":192518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":96259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52058,"num_write_kb":208232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":3989504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":787828736,"data_stored":787738411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96340,"ondisk_log_size":96340,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520692,"kb_used_data":336292,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93851148,"statfs":{"total":96636764160,"available":96103575552,"internally_reserved":0,"allocated":344363008,"data_stored":344117918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530824,"kb_used_data":358968,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93841016,"statfs":{"total":96636764160,"available":96093200384,"internally_reserved":0,"allocated":367583232,"data_stored":367334065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":181364,"kb_used_data":73316,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94190476,"statfs":{"total":96636764160,"available":96451047424,"internally_reserved":0,"allocated":75075584,"data_stored":74746931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":368238592,"data_stored":368234979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75231232,"data_stored":75148133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344358912,"data_stored":344355299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:03.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:04.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 91 20 97564 97544 4.18616 2 0.0308313 0.0186527 2024-09-05T13:14:04.442 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:04.442 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:04.475 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-05T13:14:03.535398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394273270,"num_objects":96261,"num_object_clones":0,"num_object_copies":192522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":96261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52060,"num_write_kb":208234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":975,"num_bytes_recovered":3989523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96343,"ondisk_log_size":96343,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1232880,"kb_used_data":768576,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281882640,"statfs":{"total":289910292480,"available":288647823360,"internally_reserved":0,"allocated":787021824,"data_stored":786198914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":475384838},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,3,5,8,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-343,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6621,"num_write_kb":26484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":343,"num_bytes_recovered":1404928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003650"},"pg_stats":[{"pgid":"2.f","version":"43'5944","reported_seq":6013,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.413584+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:01.413584+0000","last_peered":"2024-09-05T13:14:01.413584+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:01.413584+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5944,"log_dups_size":0,"ondisk_log_size":5944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24346624,"num_objects":5944,"num_object_clones":0,"num_object_copies":11888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":23776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6046","reported_seq":6145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:02.379175+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:02.379175+0000","last_peered":"2024-09-05T13:14:02.379175+0000","last_clean":"2024-09-05T13:14:02.379175+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:02.379175+0000","last_undegraded":"2024-09-05T13:14:02.379175+0000","last_fullsized":"2024-09-05T13:14:02.379175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24739840,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4056,"num_write_kb":16224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6063","reported_seq":6185,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.293505+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:02.293505+0000","last_peered":"2024-09-05T13:14:02.293505+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:02.293505+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6063,"log_dups_size":0,"ondisk_log_size":6063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24817664,"num_objects":6059,"num_object_clones":0,"num_object_copies":12118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3874,"num_write_kb":15496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6078","reported_seq":6139,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:02.388731+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:02.388731+0000","last_peered":"2024-09-05T13:14:02.388731+0000","last_clean":"2024-09-05T13:14:02.388731+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:02.388731+0000","last_undegraded":"2024-09-05T13:14:02.388731+0000","last_fullsized":"2024-09-05T13:14:02.388731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6078,"log_dups_size":0,"ondisk_log_size":6078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24895488,"num_objects":6078,"num_object_clones":0,"num_object_copies":12156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6078,"num_write_kb":24312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6037","reported_seq":6166,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:01.444269+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:01.444269+0000","last_peered":"2024-09-05T13:14:01.444269+0000","last_clean":"2024-09-05T13:14:01.444269+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:01.444269+0000","last_undegraded":"2024-09-05T13:14:01.444269+0000","last_fullsized":"2024-09-05T13:14:01.444269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6037,"log_dups_size":0,"ondisk_log_size":6037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6009","reported_seq":6142,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.462559+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:01.462559+0000","last_peered":"2024-09-05T13:14:01.462559+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:01.462559+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6009,"log_dups_size":0,"ondisk_log_size":6009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2464,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1553},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5962","reported_seq":6093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.511313+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:01.511313+0000","last_peered":"2024-09-05T13:14:01.511313+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:01.511313+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2281,"num_write_kb":9124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1520},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6090","reported_seq":6215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:01.521951+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:01.521951+0000","last_peered":"2024-09-05T13:14:01.521951+0000","last_clean":"2024-09-05T13:14:01.521951+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:01.521951+0000","last_undegraded":"2024-09-05T13:14:01.521951+0000","last_fullsized":"2024-09-05T13:14:01.521951+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6090,"log_dups_size":0,"ondisk_log_size":6090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24936448,"num_objects":6088,"num_object_clones":0,"num_object_copies":12176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2700,"num_write_kb":10800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6057","reported_seq":6186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.370117+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:02.370117+0000","last_peered":"2024-09-05T13:14:02.370117+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:02.370117+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6057,"log_dups_size":0,"ondisk_log_size":6057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3711,"num_write_kb":14844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6011","reported_seq":6142,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.495441+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:01.495441+0000","last_peered":"2024-09-05T13:14:01.495441+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:01.495441+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6011,"log_dups_size":0,"ondisk_log_size":6011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24588288,"num_objects":6003,"num_object_clones":0,"num_object_copies":12006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2378,"num_write_kb":9512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1549},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5965","reported_seq":8018,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.368210+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:02.368210+0000","last_peered":"2024-09-05T13:14:02.368210+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:02.368210+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5965,"log_dups_size":0,"ondisk_log_size":5965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":971,"num_bytes_recovered":3977216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3414},{"shards":"2","objects":2544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5947","reported_seq":6064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.324286+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:02.324286+0000","last_peered":"2024-09-05T13:14:02.324286+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:02.324286+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5947,"log_dups_size":0,"ondisk_log_size":5947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24334336,"num_objects":5941,"num_object_clones":0,"num_object_copies":11882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3456,"num_write_kb":13824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5960","reported_seq":6095,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.439863+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:01.439863+0000","last_peered":"2024-09-05T13:14:01.439863+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:01.439863+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5960,"log_dups_size":0,"ondisk_log_size":5960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24403968,"num_objects":5958,"num_object_clones":0,"num_object_copies":11916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2111,"num_write_kb":8444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1539},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6017","reported_seq":6199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.510984+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:01.510984+0000","last_peered":"2024-09-05T13:14:01.510984+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:01.510984+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24612864,"num_objects":6009,"num_object_clones":0,"num_object_copies":12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1916,"num_write_kb":7664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1565},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6068","reported_seq":6201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.481442+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:01.481442+0000","last_peered":"2024-09-05T13:14:01.481442+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:01.481442+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6068,"log_dups_size":0,"ondisk_log_size":6068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":8792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1498},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6086","reported_seq":6220,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.507209+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:01.507209+0000","last_peered":"2024-09-05T13:14:01.507209+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:01.507209+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6086,"log_dups_size":0,"ondisk_log_size":6086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24879587,"num_objects":6075,"num_object_clones":0,"num_object_copies":12150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2754,"num_write_kb":11016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":394273251,"num_objects":96259,"num_object_clones":0,"num_object_copies":192518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":96259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52058,"num_write_kb":208232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":3989504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":787828736,"data_stored":787738411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96340,"ondisk_log_size":96340,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520692,"kb_used_data":336292,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93851148,"statfs":{"total":96636764160,"available":96103575552,"internally_reserved":0,"allocated":344363008,"data_stored":344117918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530824,"kb_used_data":358968,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93841016,"statfs":{"total":96636764160,"available":96093200384,"internally_reserved":0,"allocated":367583232,"data_stored":367334065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":181364,"kb_used_data":73316,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94190476,"statfs":{"total":96636764160,"available":96451047424,"internally_reserved":0,"allocated":75075584,"data_stored":74746931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":368238592,"data_stored":368234979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75231232,"data_stored":75148133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344358912,"data_stored":344355299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:04.476 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:04.476 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5944", 'reported_seq': 6013, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:01.413584+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:01.413584+0000', 'last_peered': '2024-09-05T13:14:01.413584+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:01.413584+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 5944, 'log_dups_size': 0, 'ondisk_log_size': 5944, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24346624, 'num_objects': 5944, 'num_object_clones': 0, 'num_object_copies': 11888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5944, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5944, 'num_write_kb': 23776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1523}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:04.476 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:04.476 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6063", 'reported_seq': 6185, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:02.293505+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:02.293505+0000', 'last_peered': '2024-09-05T13:14:02.293505+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:02.293505+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 6063, 'log_dups_size': 0, 'ondisk_log_size': 6063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24817664, 'num_objects': 6059, 'num_object_clones': 0, 'num_object_copies': 12118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3874, 'num_write_kb': 15496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 1567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:04.476 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6009", 'reported_seq': 6142, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:01.462559+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:01.462559+0000', 'last_peered': '2024-09-05T13:14:01.462559+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:01.462559+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 6009, 'log_dups_size': 0, 'ondisk_log_size': 6009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24592384, 'num_objects': 6004, 'num_object_clones': 0, 'num_object_copies': 12008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2464, 'num_write_kb': 9856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1553}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5962", 'reported_seq': 6093, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:01.511313+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:01.511313+0000', 'last_peered': '2024-09-05T13:14:01.511313+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:01.511313+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 5962, 'log_dups_size': 0, 'ondisk_log_size': 5962, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24395776, 'num_objects': 5956, 'num_object_clones': 0, 'num_object_copies': 11912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5956, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2281, 'num_write_kb': 9124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1520}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6057", 'reported_seq': 6186, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:02.370117+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:02.370117+0000', 'last_peered': '2024-09-05T13:14:02.370117+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:02.370117+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 6057, 'log_dups_size': 0, 'ondisk_log_size': 6057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24780800, 'num_objects': 6050, 'num_object_clones': 0, 'num_object_copies': 12100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3711, 'num_write_kb': 14844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 1599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6011", 'reported_seq': 6142, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:01.495441+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:01.495441+0000', 'last_peered': '2024-09-05T13:14:01.495441+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:01.495441+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 6011, 'log_dups_size': 0, 'ondisk_log_size': 6011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24588288, 'num_objects': 6003, 'num_object_clones': 0, 'num_object_copies': 12006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2378, 'num_write_kb': 9512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1549}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5965", 'reported_seq': 8018, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:02.368210+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:02.368210+0000', 'last_peered': '2024-09-05T13:14:02.368210+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:02.368210+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 5965, 'log_dups_size': 0, 'ondisk_log_size': 5965, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24412160, 'num_objects': 5960, 'num_object_clones': 0, 'num_object_copies': 11920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3414, 'num_objects_unfound': 0, 'num_objects_dirty': 5960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3560, 'num_write_kb': 14240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 971, 'num_bytes_recovered': 3977216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3414}, {'shards': '2', 'objects': 2544}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:04.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5947", 'reported_seq': 6064, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:02.324286+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:02.324286+0000', 'last_peered': '2024-09-05T13:14:02.324286+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:02.324286+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 5947, 'log_dups_size': 0, 'ondisk_log_size': 5947, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24334336, 'num_objects': 5941, 'num_object_clones': 0, 'num_object_copies': 11882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3456, 'num_write_kb': 13824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 1554}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5960", 'reported_seq': 6095, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:01.439863+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:01.439863+0000', 'last_peered': '2024-09-05T13:14:01.439863+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:01.439863+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 5960, 'log_dups_size': 0, 'ondisk_log_size': 5960, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24403968, 'num_objects': 5958, 'num_object_clones': 0, 'num_object_copies': 11916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5958, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2111, 'num_write_kb': 8444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1539}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6017", 'reported_seq': 6199, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:01.510984+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:01.510984+0000', 'last_peered': '2024-09-05T13:14:01.510984+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:01.510984+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 6017, 'log_dups_size': 0, 'ondisk_log_size': 6017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24612864, 'num_objects': 6009, 'num_object_clones': 0, 'num_object_copies': 12018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1916, 'num_write_kb': 7664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1565}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6068", 'reported_seq': 6201, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:01.481442+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:01.481442+0000', 'last_peered': '2024-09-05T13:14:01.481442+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:01.481442+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 6068, 'log_dups_size': 0, 'ondisk_log_size': 6068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24838144, 'num_objects': 6064, 'num_object_clones': 0, 'num_object_copies': 12128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2198, 'num_write_kb': 8792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1498}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:04.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6086", 'reported_seq': 6220, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:01.507209+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:01.507209+0000', 'last_peered': '2024-09-05T13:14:01.507209+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:01.507209+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 6086, 'log_dups_size': 0, 'ondisk_log_size': 6086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24879587, 'num_objects': 6075, 'num_object_clones': 0, 'num_object_copies': 12150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6075, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2754, 'num_write_kb': 11016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1510}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:04.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:05.173 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:05.173 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/192522 objects degraded (25.508%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":172,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":96261,"data_bytes":394273270,"bytes_used":1262469120,"bytes_avail":288647823360,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":192522,"degraded_ratio":0.25507734181028663,"misplaced_objects":3414,"misplaced_total":192522,"misplaced_ratio":0.017733038302116123,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":117041,"recovering_keys_per_sec":0,"num_objects_recovered":343,"num_bytes_recovered":1404928,"num_keys_recovered":0,"write_bytes_sec":2259280,"read_op_per_sec":0,"write_op_per_sec":551},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:14:05.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:05.204 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:05.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 92 20 98069 98049 4.1621 1.97266 0.0276618 0.0187616 2024-09-05T13:14:05.776 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:05.777 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:05.800 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-05T13:14:05.536018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394273270,"num_objects":96261,"num_object_clones":0,"num_object_copies":192522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":96261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52060,"num_write_kb":208234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":975,"num_bytes_recovered":3989523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96343,"ondisk_log_size":96343,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1263656,"kb_used_data":777144,"kb_used_omap":12,"kb_used_meta":486451,"kb_avail":281851864,"statfs":{"total":289910292480,"available":288616308736,"internally_reserved":0,"allocated":795795456,"data_stored":794972546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":498125830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,4,2,8,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-226,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5570,"num_write_kb":22280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":925696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003535"},"pg_stats":[{"pgid":"2.f","version":"43'5944","reported_seq":6013,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.413584+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:01.413584+0000","last_peered":"2024-09-05T13:14:01.413584+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:01.413584+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":5944,"log_dups_size":0,"ondisk_log_size":5944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24346624,"num_objects":5944,"num_object_clones":0,"num_object_copies":11888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":23776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6046","reported_seq":6145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:02.379175+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:02.379175+0000","last_peered":"2024-09-05T13:14:02.379175+0000","last_clean":"2024-09-05T13:14:02.379175+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:02.379175+0000","last_undegraded":"2024-09-05T13:14:02.379175+0000","last_fullsized":"2024-09-05T13:14:02.379175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24739840,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4056,"num_write_kb":16224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6063","reported_seq":6185,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.293505+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:02.293505+0000","last_peered":"2024-09-05T13:14:02.293505+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:02.293505+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6063,"log_dups_size":0,"ondisk_log_size":6063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24817664,"num_objects":6059,"num_object_clones":0,"num_object_copies":12118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3874,"num_write_kb":15496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6078","reported_seq":6139,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:02.388731+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:02.388731+0000","last_peered":"2024-09-05T13:14:02.388731+0000","last_clean":"2024-09-05T13:14:02.388731+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:02.388731+0000","last_undegraded":"2024-09-05T13:14:02.388731+0000","last_fullsized":"2024-09-05T13:14:02.388731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6078,"log_dups_size":0,"ondisk_log_size":6078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24895488,"num_objects":6078,"num_object_clones":0,"num_object_copies":12156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6078,"num_write_kb":24312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6037","reported_seq":6166,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:01.444269+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:01.444269+0000","last_peered":"2024-09-05T13:14:01.444269+0000","last_clean":"2024-09-05T13:14:01.444269+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:01.444269+0000","last_undegraded":"2024-09-05T13:14:01.444269+0000","last_fullsized":"2024-09-05T13:14:01.444269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6037,"log_dups_size":0,"ondisk_log_size":6037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6009","reported_seq":6142,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.462559+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:01.462559+0000","last_peered":"2024-09-05T13:14:01.462559+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:01.462559+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6009,"log_dups_size":0,"ondisk_log_size":6009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2464,"num_write_kb":9856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1553},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5962","reported_seq":6093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.511313+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:01.511313+0000","last_peered":"2024-09-05T13:14:01.511313+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:01.511313+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":5962,"log_dups_size":0,"ondisk_log_size":5962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2281,"num_write_kb":9124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1520},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6090","reported_seq":6215,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:01.521951+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:01.521951+0000","last_peered":"2024-09-05T13:14:01.521951+0000","last_clean":"2024-09-05T13:14:01.521951+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:01.521951+0000","last_undegraded":"2024-09-05T13:14:01.521951+0000","last_fullsized":"2024-09-05T13:14:01.521951+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6090,"log_dups_size":0,"ondisk_log_size":6090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24936448,"num_objects":6088,"num_object_clones":0,"num_object_copies":12176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2700,"num_write_kb":10800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6057","reported_seq":6186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.370117+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:02.370117+0000","last_peered":"2024-09-05T13:14:02.370117+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:02.370117+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6057,"log_dups_size":0,"ondisk_log_size":6057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24780800,"num_objects":6050,"num_object_clones":0,"num_object_copies":12100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3711,"num_write_kb":14844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6011","reported_seq":6142,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.495441+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:01.495441+0000","last_peered":"2024-09-05T13:14:01.495441+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:01.495441+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6011,"log_dups_size":0,"ondisk_log_size":6011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24588288,"num_objects":6003,"num_object_clones":0,"num_object_copies":12006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2378,"num_write_kb":9512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1549},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.017160+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.017160+0000","last_peered":"2024-09-05T13:13:18.017160+0000","last_clean":"2024-09-05T13:13:18.017160+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.017160+0000","last_undegraded":"2024-09-05T13:13:18.017160+0000","last_fullsized":"2024-09-05T13:13:18.017160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5965","reported_seq":8018,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.368210+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:02.368210+0000","last_peered":"2024-09-05T13:14:02.368210+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:02.368210+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":5965,"log_dups_size":0,"ondisk_log_size":5965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":971,"num_bytes_recovered":3977216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3414},{"shards":"2","objects":2544}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5947","reported_seq":6064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:02.324286+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:02.324286+0000","last_peered":"2024-09-05T13:14:02.324286+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:02.324286+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":5947,"log_dups_size":0,"ondisk_log_size":5947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24334336,"num_objects":5941,"num_object_clones":0,"num_object_copies":11882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3456,"num_write_kb":13824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5960","reported_seq":6095,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.439863+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:01.439863+0000","last_peered":"2024-09-05T13:14:01.439863+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:01.439863+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":5960,"log_dups_size":0,"ondisk_log_size":5960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24403968,"num_objects":5958,"num_object_clones":0,"num_object_copies":11916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2111,"num_write_kb":8444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1539},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6017","reported_seq":6199,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.510984+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:01.510984+0000","last_peered":"2024-09-05T13:14:01.510984+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:01.510984+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24612864,"num_objects":6009,"num_object_clones":0,"num_object_copies":12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1916,"num_write_kb":7664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1565},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6068","reported_seq":6201,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.481442+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:01.481442+0000","last_peered":"2024-09-05T13:14:01.481442+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:01.481442+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6068,"log_dups_size":0,"ondisk_log_size":6068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2198,"num_write_kb":8792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1498},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6086","reported_seq":6220,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:01.507209+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:01.507209+0000","last_peered":"2024-09-05T13:14:01.507209+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:01.507209+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6086,"log_dups_size":0,"ondisk_log_size":6086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24879587,"num_objects":6075,"num_object_clones":0,"num_object_copies":12150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2754,"num_write_kb":11016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":394273251,"num_objects":96259,"num_object_clones":0,"num_object_copies":192518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3414,"num_objects_unfound":0,"num_objects_dirty":96259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52058,"num_write_kb":208232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":974,"num_bytes_recovered":3989504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":796741632,"data_stored":796651307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96340,"ondisk_log_size":96340,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520692,"kb_used_data":336292,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93851148,"statfs":{"total":96636764160,"available":96103575552,"internally_reserved":0,"allocated":344363008,"data_stored":344117918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530824,"kb_used_data":358968,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93841016,"statfs":{"total":96636764160,"available":96093200384,"internally_reserved":0,"allocated":367583232,"data_stored":367334065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212140,"kb_used_data":81884,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94159700,"statfs":{"total":96636764160,"available":96419532800,"internally_reserved":0,"allocated":83849216,"data_stored":83520563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133361327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":368238592,"data_stored":368234979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":84144128,"data_stored":84061029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344358912,"data_stored":344355299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:06.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 93 20 98591 98571 4.13927 2.03906 0.0125105 0.0188654 2024-09-05T13:14:07.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 94 20 99172 99152 4.11938 2.26953 0.0240802 0.0189589 2024-09-05T13:14:08.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 95 20 99741 99721 4.09942 2.22266 0.0140104 0.0190503 2024-09-05T13:14:08.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:09.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 96 20 100272 100252 4.07832 2.07422 0.0732674 0.019147 2024-09-05T13:14:09.358 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:09.360 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:09.384 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-05T13:14:07.536436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405336566,"num_objects":98962,"num_object_clones":0,"num_object_copies":197924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":98962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54761,"num_write_kb":219038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1095,"num_bytes_recovered":4481043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99044,"ondisk_log_size":99044,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1285200,"kb_used_data":790368,"kb_used_omap":12,"kb_used_meta":494771,"kb_avail":281830320,"statfs":{"total":289910292480,"available":288594247680,"internally_reserved":0,"allocated":809336832,"data_stored":808513922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":506645510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,4,4,12,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33878016,"num_objects":8271,"num_object_clones":0,"num_object_copies":16542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-346,"num_objects_unfound":0,"num_objects_dirty":8271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8271,"num_write_kb":33084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":1417216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003354"},"pg_stats":[{"pgid":"2.f","version":"43'6125","reported_seq":6193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.495158+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:06.495158+0000","last_peered":"2024-09-05T13:14:06.495158+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:06.495158+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6125,"log_dups_size":0,"ondisk_log_size":6125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25088000,"num_objects":6125,"num_object_clones":0,"num_object_copies":12250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6125,"num_write_kb":24500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1704},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6194","reported_seq":6293,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:07.371082+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:07.371082+0000","last_peered":"2024-09-05T13:14:07.371082+0000","last_clean":"2024-09-05T13:14:07.371082+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:07.371082+0000","last_undegraded":"2024-09-05T13:14:07.371082+0000","last_fullsized":"2024-09-05T13:14:07.371082+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6194,"log_dups_size":0,"ondisk_log_size":6194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4204,"num_write_kb":16816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6226","reported_seq":6348,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.326030+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:07.326030+0000","last_peered":"2024-09-05T13:14:07.326030+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:07.326030+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6226,"log_dups_size":0,"ondisk_log_size":6226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485312,"num_objects":6222,"num_object_clones":0,"num_object_copies":12444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4037,"num_write_kb":16148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1730}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6261","reported_seq":6325,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:07.356740+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:07.356740+0000","last_peered":"2024-09-05T13:14:07.356740+0000","last_clean":"2024-09-05T13:14:07.356740+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:07.356740+0000","last_undegraded":"2024-09-05T13:14:07.356740+0000","last_fullsized":"2024-09-05T13:14:07.356740+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25645056,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6261,"num_write_kb":25044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6222","reported_seq":6351,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:06.544989+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:06.544989+0000","last_peered":"2024-09-05T13:14:06.544989+0000","last_clean":"2024-09-05T13:14:06.544989+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:06.544989+0000","last_undegraded":"2024-09-05T13:14:06.544989+0000","last_fullsized":"2024-09-05T13:14:06.544989+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6222,"log_dups_size":0,"ondisk_log_size":6222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25456640,"num_objects":6215,"num_object_clones":0,"num_object_copies":12430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2762,"num_write_kb":11048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6195","reported_seq":6328,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.484809+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:06.484809+0000","last_peered":"2024-09-05T13:14:06.484809+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:06.484809+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6195,"log_dups_size":0,"ondisk_log_size":6195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25354240,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2650,"num_write_kb":10600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1739},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6130","reported_seq":6264,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.530282+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:06.530282+0000","last_peered":"2024-09-05T13:14:06.530282+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:06.530282+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6130,"log_dups_size":0,"ondisk_log_size":6130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25083904,"num_objects":6124,"num_object_clones":0,"num_object_copies":12248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2449,"num_write_kb":9796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1688},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6255","reported_seq":6380,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:06.466962+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:06.466962+0000","last_peered":"2024-09-05T13:14:06.466962+0000","last_clean":"2024-09-05T13:14:06.466962+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:06.466962+0000","last_undegraded":"2024-09-05T13:14:06.466962+0000","last_fullsized":"2024-09-05T13:14:06.466962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6255,"log_dups_size":0,"ondisk_log_size":6255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25612288,"num_objects":6253,"num_object_clones":0,"num_object_copies":12506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2865,"num_write_kb":11460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6216","reported_seq":6345,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.360800+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:07.360800+0000","last_peered":"2024-09-05T13:14:07.360800+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:07.360800+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6216,"log_dups_size":0,"ondisk_log_size":6216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3870,"num_write_kb":15480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1758}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6161","reported_seq":6290,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.541073+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:06.541073+0000","last_peered":"2024-09-05T13:14:06.541073+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:06.541073+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6161,"log_dups_size":0,"ondisk_log_size":6161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25202688,"num_objects":6153,"num_object_clones":0,"num_object_copies":12306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2528,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1699},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6129","reported_seq":8421,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.394673+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:07.394673+0000","last_peered":"2024-09-05T13:14:07.394673+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:07.394673+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6129,"log_dups_size":0,"ondisk_log_size":6129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25083904,"num_objects":6124,"num_object_clones":0,"num_object_copies":12248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":6124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3724,"num_write_kb":14896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":4468736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3294},{"shards":"2","objects":2828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6099","reported_seq":6215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.293184+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:07.293184+0000","last_peered":"2024-09-05T13:14:07.293184+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:07.293184+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6099,"log_dups_size":0,"ondisk_log_size":6099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3608,"num_write_kb":14432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6137","reported_seq":6272,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.494813+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:06.494813+0000","last_peered":"2024-09-05T13:14:06.494813+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:06.494813+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6137,"log_dups_size":0,"ondisk_log_size":6137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25128960,"num_objects":6135,"num_object_clones":0,"num_object_copies":12270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1716},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6176","reported_seq":6358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.513398+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:06.513398+0000","last_peered":"2024-09-05T13:14:06.513398+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:06.513398+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6176,"log_dups_size":0,"ondisk_log_size":6176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2075,"num_write_kb":8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1724},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6245","reported_seq":6377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.542652+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:06.542652+0000","last_peered":"2024-09-05T13:14:06.542652+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:06.542652+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6245,"log_dups_size":0,"ondisk_log_size":6245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25563136,"num_objects":6241,"num_object_clones":0,"num_object_copies":12482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1675},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6270","reported_seq":6404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.470398+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:06.470398+0000","last_peered":"2024-09-05T13:14:06.470398+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:06.470398+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6270,"log_dups_size":0,"ondisk_log_size":6270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25633251,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2938,"num_write_kb":11752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1694},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405336547,"num_objects":98960,"num_object_clones":0,"num_object_copies":197920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":98960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54759,"num_write_kb":219036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1094,"num_bytes_recovered":4481024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":810622976,"data_stored":810532651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99041,"ondisk_log_size":99041,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530152,"kb_used_data":341592,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93841688,"statfs":{"total":96636764160,"available":96093888512,"internally_reserved":0,"allocated":349790208,"data_stored":349545118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":193064621},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542908,"kb_used_data":366892,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93828932,"statfs":{"total":96636764160,"available":96080826368,"internally_reserved":0,"allocated":375697408,"data_stored":375448241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,1,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212140,"kb_used_data":81884,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94159700,"statfs":{"total":96636764160,"available":96419532800,"internally_reserved":0,"allocated":83849216,"data_stored":83520563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133361327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":376684544,"data_stored":376680931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":84144128,"data_stored":84061029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349794304,"data_stored":349790691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:09.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:09.923 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:09.924 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:09.949 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-05T13:14:09.537297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405336566,"num_objects":98962,"num_object_clones":0,"num_object_copies":197924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":98962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54761,"num_write_kb":219038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1095,"num_bytes_recovered":4481043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99044,"ondisk_log_size":99044,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1285580,"kb_used_data":799068,"kb_used_omap":12,"kb_used_meta":486451,"kb_avail":281829940,"statfs":{"total":289910292480,"available":288593858560,"internally_reserved":0,"allocated":818245632,"data_stored":817422722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":498125830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,4,5,14,17],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22294528,"num_objects":5443,"num_object_clones":0,"num_object_copies":10886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-236,"num_objects_unfound":0,"num_objects_dirty":5443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5443,"num_write_kb":21772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003674"},"pg_stats":[{"pgid":"2.f","version":"43'6125","reported_seq":6193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.495158+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:06.495158+0000","last_peered":"2024-09-05T13:14:06.495158+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:06.495158+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6125,"log_dups_size":0,"ondisk_log_size":6125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25088000,"num_objects":6125,"num_object_clones":0,"num_object_copies":12250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6125,"num_write_kb":24500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1704},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6194","reported_seq":6293,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:07.371082+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:07.371082+0000","last_peered":"2024-09-05T13:14:07.371082+0000","last_clean":"2024-09-05T13:14:07.371082+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:07.371082+0000","last_undegraded":"2024-09-05T13:14:07.371082+0000","last_fullsized":"2024-09-05T13:14:07.371082+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6194,"log_dups_size":0,"ondisk_log_size":6194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4204,"num_write_kb":16816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6226","reported_seq":6348,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.326030+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:07.326030+0000","last_peered":"2024-09-05T13:14:07.326030+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:07.326030+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6226,"log_dups_size":0,"ondisk_log_size":6226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485312,"num_objects":6222,"num_object_clones":0,"num_object_copies":12444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4037,"num_write_kb":16148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1730}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6261","reported_seq":6325,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:07.356740+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:07.356740+0000","last_peered":"2024-09-05T13:14:07.356740+0000","last_clean":"2024-09-05T13:14:07.356740+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:07.356740+0000","last_undegraded":"2024-09-05T13:14:07.356740+0000","last_fullsized":"2024-09-05T13:14:07.356740+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25645056,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6261,"num_write_kb":25044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6222","reported_seq":6351,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:06.544989+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:06.544989+0000","last_peered":"2024-09-05T13:14:06.544989+0000","last_clean":"2024-09-05T13:14:06.544989+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:06.544989+0000","last_undegraded":"2024-09-05T13:14:06.544989+0000","last_fullsized":"2024-09-05T13:14:06.544989+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6222,"log_dups_size":0,"ondisk_log_size":6222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25456640,"num_objects":6215,"num_object_clones":0,"num_object_copies":12430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2762,"num_write_kb":11048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6195","reported_seq":6328,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.484809+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:06.484809+0000","last_peered":"2024-09-05T13:14:06.484809+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:06.484809+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6195,"log_dups_size":0,"ondisk_log_size":6195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25354240,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2650,"num_write_kb":10600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1739},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6130","reported_seq":6264,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.530282+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:06.530282+0000","last_peered":"2024-09-05T13:14:06.530282+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:06.530282+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6130,"log_dups_size":0,"ondisk_log_size":6130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25083904,"num_objects":6124,"num_object_clones":0,"num_object_copies":12248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2449,"num_write_kb":9796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1688},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6255","reported_seq":6380,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:06.466962+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:06.466962+0000","last_peered":"2024-09-05T13:14:06.466962+0000","last_clean":"2024-09-05T13:14:06.466962+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:06.466962+0000","last_undegraded":"2024-09-05T13:14:06.466962+0000","last_fullsized":"2024-09-05T13:14:06.466962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6255,"log_dups_size":0,"ondisk_log_size":6255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25612288,"num_objects":6253,"num_object_clones":0,"num_object_copies":12506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2865,"num_write_kb":11460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6216","reported_seq":6345,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.360800+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:07.360800+0000","last_peered":"2024-09-05T13:14:07.360800+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:07.360800+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6216,"log_dups_size":0,"ondisk_log_size":6216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3870,"num_write_kb":15480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1758}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6161","reported_seq":6290,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.541073+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:06.541073+0000","last_peered":"2024-09-05T13:14:06.541073+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:06.541073+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6161,"log_dups_size":0,"ondisk_log_size":6161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25202688,"num_objects":6153,"num_object_clones":0,"num_object_copies":12306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2528,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1699},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6129","reported_seq":8421,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.394673+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:07.394673+0000","last_peered":"2024-09-05T13:14:07.394673+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:07.394673+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6129,"log_dups_size":0,"ondisk_log_size":6129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25083904,"num_objects":6124,"num_object_clones":0,"num_object_copies":12248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":6124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3724,"num_write_kb":14896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":4468736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3294},{"shards":"2","objects":2828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6099","reported_seq":6215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.293184+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:07.293184+0000","last_peered":"2024-09-05T13:14:07.293184+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:07.293184+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6099,"log_dups_size":0,"ondisk_log_size":6099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3608,"num_write_kb":14432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6137","reported_seq":6272,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.494813+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:06.494813+0000","last_peered":"2024-09-05T13:14:06.494813+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:06.494813+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6137,"log_dups_size":0,"ondisk_log_size":6137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25128960,"num_objects":6135,"num_object_clones":0,"num_object_copies":12270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1716},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6176","reported_seq":6358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.513398+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:06.513398+0000","last_peered":"2024-09-05T13:14:06.513398+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:06.513398+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6176,"log_dups_size":0,"ondisk_log_size":6176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2075,"num_write_kb":8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1724},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6245","reported_seq":6377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.542652+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:06.542652+0000","last_peered":"2024-09-05T13:14:06.542652+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:06.542652+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6245,"log_dups_size":0,"ondisk_log_size":6245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25563136,"num_objects":6241,"num_object_clones":0,"num_object_copies":12482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1675},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6270","reported_seq":6404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.470398+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:06.470398+0000","last_peered":"2024-09-05T13:14:06.470398+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:06.470398+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6270,"log_dups_size":0,"ondisk_log_size":6270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25633251,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2938,"num_write_kb":11752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1694},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405336547,"num_objects":98960,"num_object_clones":0,"num_object_copies":197920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":98960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54759,"num_write_kb":219036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1094,"num_bytes_recovered":4481024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":819462144,"data_stored":819371819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99041,"ondisk_log_size":99041,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530152,"kb_used_data":341592,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93841688,"statfs":{"total":96636764160,"available":96093888512,"internally_reserved":0,"allocated":349790208,"data_stored":349545118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":193064621},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542908,"kb_used_data":366892,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93828932,"statfs":{"total":96636764160,"available":96080826368,"internally_reserved":0,"allocated":375697408,"data_stored":375448241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,1,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212520,"kb_used_data":90584,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94159320,"statfs":{"total":96636764160,"available":96419143680,"internally_reserved":0,"allocated":92758016,"data_stored":92429363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":124841647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,4,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":376684544,"data_stored":376680931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92983296,"data_stored":92900197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349794304,"data_stored":349790691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:09.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:09.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6125", 'reported_seq': 6193, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:06.495158+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:06.495158+0000', 'last_peered': '2024-09-05T13:14:06.495158+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:06.495158+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 6125, 'log_dups_size': 0, 'ondisk_log_size': 6125, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25088000, 'num_objects': 6125, 'num_object_clones': 0, 'num_object_copies': 12250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6125, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6125, 'num_write_kb': 24500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1704}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:09.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:09.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6226", 'reported_seq': 6348, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:07.326030+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:07.326030+0000', 'last_peered': '2024-09-05T13:14:07.326030+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:07.326030+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 6226, 'log_dups_size': 0, 'ondisk_log_size': 6226, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25485312, 'num_objects': 6222, 'num_object_clones': 0, 'num_object_copies': 12444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4037, 'num_write_kb': 16148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 1730}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:09.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:09.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6195", 'reported_seq': 6328, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:06.484809+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:06.484809+0000', 'last_peered': '2024-09-05T13:14:06.484809+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:06.484809+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 6195, 'log_dups_size': 0, 'ondisk_log_size': 6195, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25354240, 'num_objects': 6190, 'num_object_clones': 0, 'num_object_copies': 12380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2650, 'num_write_kb': 10600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1739}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:09.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:09.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6130", 'reported_seq': 6264, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:06.530282+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:06.530282+0000', 'last_peered': '2024-09-05T13:14:06.530282+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:06.530282+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 6130, 'log_dups_size': 0, 'ondisk_log_size': 6130, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25083904, 'num_objects': 6124, 'num_object_clones': 0, 'num_object_copies': 12248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2449, 'num_write_kb': 9796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1688}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:09.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:09.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6216", 'reported_seq': 6345, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:07.360800+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:07.360800+0000', 'last_peered': '2024-09-05T13:14:07.360800+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:07.360800+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 6216, 'log_dups_size': 0, 'ondisk_log_size': 6216, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25432064, 'num_objects': 6209, 'num_object_clones': 0, 'num_object_copies': 12418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3870, 'num_write_kb': 15480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 1758}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6161", 'reported_seq': 6290, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:06.541073+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:06.541073+0000', 'last_peered': '2024-09-05T13:14:06.541073+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:06.541073+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 6161, 'log_dups_size': 0, 'ondisk_log_size': 6161, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25202688, 'num_objects': 6153, 'num_object_clones': 0, 'num_object_copies': 12306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6153, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2528, 'num_write_kb': 10112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1699}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6129", 'reported_seq': 8421, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:07.394673+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:07.394673+0000', 'last_peered': '2024-09-05T13:14:07.394673+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:07.394673+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 6129, 'log_dups_size': 0, 'ondisk_log_size': 6129, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25083904, 'num_objects': 6124, 'num_object_clones': 0, 'num_object_copies': 12248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3294, 'num_objects_unfound': 0, 'num_objects_dirty': 6124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3724, 'num_write_kb': 14896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1091, 'num_bytes_recovered': 4468736, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3294}, {'shards': '2', 'objects': 2828}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6099", 'reported_seq': 6215, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:07.293184+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:07.293184+0000', 'last_peered': '2024-09-05T13:14:07.293184+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:07.293184+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 6099, 'log_dups_size': 0, 'ondisk_log_size': 6099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24956928, 'num_objects': 6093, 'num_object_clones': 0, 'num_object_copies': 12186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3608, 'num_write_kb': 14432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 1706}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6137", 'reported_seq': 6272, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:06.494813+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:06.494813+0000', 'last_peered': '2024-09-05T13:14:06.494813+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:06.494813+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 6137, 'log_dups_size': 0, 'ondisk_log_size': 6137, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25128960, 'num_objects': 6135, 'num_object_clones': 0, 'num_object_copies': 12270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2288, 'num_write_kb': 9152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1716}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:09.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6176", 'reported_seq': 6358, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:06.513398+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:06.513398+0000', 'last_peered': '2024-09-05T13:14:06.513398+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:06.513398+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 6176, 'log_dups_size': 0, 'ondisk_log_size': 6176, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25264128, 'num_objects': 6168, 'num_object_clones': 0, 'num_object_copies': 12336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2075, 'num_write_kb': 8300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1724}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:09.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:09.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6245", 'reported_seq': 6377, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:06.542652+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:06.542652+0000', 'last_peered': '2024-09-05T13:14:06.542652+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:06.542652+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 6245, 'log_dups_size': 0, 'ondisk_log_size': 6245, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25563136, 'num_objects': 6241, 'num_object_clones': 0, 'num_object_copies': 12482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6241, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2375, 'num_write_kb': 9500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1675}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:09.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:09.953 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6270", 'reported_seq': 6404, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:06.470398+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:06.470398+0000', 'last_peered': '2024-09-05T13:14:06.470398+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:06.470398+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 6270, 'log_dups_size': 0, 'ondisk_log_size': 6270, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25633251, 'num_objects': 6259, 'num_object_clones': 0, 'num_object_copies': 12518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2938, 'num_write_kb': 11752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1694}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:09.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:10.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 97 20 100769 100749 4.05629 1.94141 0.022354 0.0192511 2024-09-05T13:14:10.648 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:10.648 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/197924 objects degraded (24.812%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":177,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98962,"data_bytes":405336566,"bytes_used":1316433920,"bytes_avail":288593858560,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":197924,"degraded_ratio":0.24811543824902485,"misplaced_objects":3294,"misplaced_total":197924,"misplaced_ratio":0.016642751763303085,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":80530,"recovering_keys_per_sec":0,"num_objects_recovered":236,"num_bytes_recovered":966656,"num_keys_recovered":0,"write_bytes_sec":1857308,"read_op_per_sec":0,"write_op_per_sec":453},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:14:10.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:10.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:11.198 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:11.198 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:11.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 98 20 101266 101246 4.03471 1.94141 0.0159036 0.0193546 2024-09-05T13:14:11.227 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-05T13:14:09.537297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405336566,"num_objects":98962,"num_object_clones":0,"num_object_copies":197924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":98962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54761,"num_write_kb":219038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1095,"num_bytes_recovered":4481043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99044,"ondisk_log_size":99044,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1285580,"kb_used_data":799068,"kb_used_omap":12,"kb_used_meta":486451,"kb_avail":281829940,"statfs":{"total":289910292480,"available":288593858560,"internally_reserved":0,"allocated":818245632,"data_stored":817422722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":498125830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,4,5,14,17],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22294528,"num_objects":5443,"num_object_clones":0,"num_object_copies":10886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-236,"num_objects_unfound":0,"num_objects_dirty":5443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5443,"num_write_kb":21772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003674"},"pg_stats":[{"pgid":"2.f","version":"43'6125","reported_seq":6193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.495158+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:06.495158+0000","last_peered":"2024-09-05T13:14:06.495158+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:06.495158+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6125,"log_dups_size":0,"ondisk_log_size":6125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25088000,"num_objects":6125,"num_object_clones":0,"num_object_copies":12250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6125,"num_write_kb":24500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1704},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6194","reported_seq":6293,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:07.371082+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:07.371082+0000","last_peered":"2024-09-05T13:14:07.371082+0000","last_clean":"2024-09-05T13:14:07.371082+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:07.371082+0000","last_undegraded":"2024-09-05T13:14:07.371082+0000","last_fullsized":"2024-09-05T13:14:07.371082+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6194,"log_dups_size":0,"ondisk_log_size":6194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4204,"num_write_kb":16816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6226","reported_seq":6348,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.326030+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:07.326030+0000","last_peered":"2024-09-05T13:14:07.326030+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:07.326030+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6226,"log_dups_size":0,"ondisk_log_size":6226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485312,"num_objects":6222,"num_object_clones":0,"num_object_copies":12444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4037,"num_write_kb":16148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1730}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6261","reported_seq":6325,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:07.356740+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:07.356740+0000","last_peered":"2024-09-05T13:14:07.356740+0000","last_clean":"2024-09-05T13:14:07.356740+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:07.356740+0000","last_undegraded":"2024-09-05T13:14:07.356740+0000","last_fullsized":"2024-09-05T13:14:07.356740+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25645056,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6261,"num_write_kb":25044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6222","reported_seq":6351,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:06.544989+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:06.544989+0000","last_peered":"2024-09-05T13:14:06.544989+0000","last_clean":"2024-09-05T13:14:06.544989+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:06.544989+0000","last_undegraded":"2024-09-05T13:14:06.544989+0000","last_fullsized":"2024-09-05T13:14:06.544989+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6222,"log_dups_size":0,"ondisk_log_size":6222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25456640,"num_objects":6215,"num_object_clones":0,"num_object_copies":12430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2762,"num_write_kb":11048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6195","reported_seq":6328,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.484809+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:06.484809+0000","last_peered":"2024-09-05T13:14:06.484809+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:06.484809+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6195,"log_dups_size":0,"ondisk_log_size":6195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25354240,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2650,"num_write_kb":10600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1739},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6130","reported_seq":6264,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.530282+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:06.530282+0000","last_peered":"2024-09-05T13:14:06.530282+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:06.530282+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6130,"log_dups_size":0,"ondisk_log_size":6130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25083904,"num_objects":6124,"num_object_clones":0,"num_object_copies":12248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2449,"num_write_kb":9796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1688},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6255","reported_seq":6380,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:06.466962+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:06.466962+0000","last_peered":"2024-09-05T13:14:06.466962+0000","last_clean":"2024-09-05T13:14:06.466962+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:06.466962+0000","last_undegraded":"2024-09-05T13:14:06.466962+0000","last_fullsized":"2024-09-05T13:14:06.466962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6255,"log_dups_size":0,"ondisk_log_size":6255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25612288,"num_objects":6253,"num_object_clones":0,"num_object_copies":12506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2865,"num_write_kb":11460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6216","reported_seq":6345,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.360800+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:07.360800+0000","last_peered":"2024-09-05T13:14:07.360800+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:07.360800+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6216,"log_dups_size":0,"ondisk_log_size":6216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3870,"num_write_kb":15480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1758}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6161","reported_seq":6290,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.541073+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:06.541073+0000","last_peered":"2024-09-05T13:14:06.541073+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:06.541073+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6161,"log_dups_size":0,"ondisk_log_size":6161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25202688,"num_objects":6153,"num_object_clones":0,"num_object_copies":12306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2528,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1699},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6129","reported_seq":8421,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.394673+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:07.394673+0000","last_peered":"2024-09-05T13:14:07.394673+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:07.394673+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6129,"log_dups_size":0,"ondisk_log_size":6129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25083904,"num_objects":6124,"num_object_clones":0,"num_object_copies":12248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":6124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3724,"num_write_kb":14896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1091,"num_bytes_recovered":4468736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3294},{"shards":"2","objects":2828}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6099","reported_seq":6215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:07.293184+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:07.293184+0000","last_peered":"2024-09-05T13:14:07.293184+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:07.293184+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6099,"log_dups_size":0,"ondisk_log_size":6099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24956928,"num_objects":6093,"num_object_clones":0,"num_object_copies":12186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3608,"num_write_kb":14432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1706}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6137","reported_seq":6272,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.494813+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:06.494813+0000","last_peered":"2024-09-05T13:14:06.494813+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:06.494813+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6137,"log_dups_size":0,"ondisk_log_size":6137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25128960,"num_objects":6135,"num_object_clones":0,"num_object_copies":12270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1716},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6176","reported_seq":6358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.513398+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:06.513398+0000","last_peered":"2024-09-05T13:14:06.513398+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:06.513398+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6176,"log_dups_size":0,"ondisk_log_size":6176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2075,"num_write_kb":8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1724},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6245","reported_seq":6377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.542652+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:06.542652+0000","last_peered":"2024-09-05T13:14:06.542652+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:06.542652+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6245,"log_dups_size":0,"ondisk_log_size":6245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25563136,"num_objects":6241,"num_object_clones":0,"num_object_copies":12482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1675},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6270","reported_seq":6404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:06.470398+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:06.470398+0000","last_peered":"2024-09-05T13:14:06.470398+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:06.470398+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6270,"log_dups_size":0,"ondisk_log_size":6270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25633251,"num_objects":6259,"num_object_clones":0,"num_object_copies":12518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2938,"num_write_kb":11752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1694},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405336547,"num_objects":98960,"num_object_clones":0,"num_object_copies":197920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3294,"num_objects_unfound":0,"num_objects_dirty":98960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54759,"num_write_kb":219036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1094,"num_bytes_recovered":4481024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":819462144,"data_stored":819371819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99041,"ondisk_log_size":99041,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530152,"kb_used_data":341592,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93841688,"statfs":{"total":96636764160,"available":96093888512,"internally_reserved":0,"allocated":349790208,"data_stored":349545118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":193064621},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542908,"kb_used_data":366892,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93828932,"statfs":{"total":96636764160,"available":96080826368,"internally_reserved":0,"allocated":375697408,"data_stored":375448241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,1,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212520,"kb_used_data":90584,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94159320,"statfs":{"total":96636764160,"available":96419143680,"internally_reserved":0,"allocated":92758016,"data_stored":92429363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":124841647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,4,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":376684544,"data_stored":376680931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92983296,"data_stored":92900197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":349794304,"data_stored":349790691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:12.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 99 20 101816 101796 4.01566 2.14844 0.0777164 0.019446 2024-09-05T13:14:13.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:14:13.207557+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0195371 2024-09-05T13:14:13.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:14:13.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 100 20 102373 102353 3.99726 2.17578 0.0312139 0.0195371 2024-09-05T13:14:14.209 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 101 20 102934 102914 3.97931 2.19141 0.0805005 0.019625 2024-09-05T13:14:14.229 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:14.796 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:14.821 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-05T13:14:13.538749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":416268790,"num_objects":101631,"num_object_clones":0,"num_object_copies":203262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":101631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57430,"num_write_kb":229714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1214,"num_bytes_recovered":4968467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101713,"ondisk_log_size":101713,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1308920,"kb_used_data":812680,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281806600,"statfs":{"total":289910292480,"available":288569958400,"internally_reserved":0,"allocated":832184320,"data_stored":831361410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":508087302},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,7,7,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26214400,"num_objects":6400,"num_object_clones":0,"num_object_copies":12800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-355,"num_objects_unfound":0,"num_objects_dirty":6400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6400,"num_write_kb":25600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004130"},"pg_stats":[{"pgid":"2.f","version":"43'6287","reported_seq":6352,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.381387+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:11.381387+0000","last_peered":"2024-09-05T13:14:11.381387+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:11.381387+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25751552,"num_objects":6287,"num_object_clones":0,"num_object_copies":12574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1866},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6360","reported_seq":6458,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:12.386641+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:12.386641+0000","last_peered":"2024-09-05T13:14:12.386641+0000","last_clean":"2024-09-05T13:14:12.386641+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:12.386641+0000","last_undegraded":"2024-09-05T13:14:12.386641+0000","last_fullsized":"2024-09-05T13:14:12.386641+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6360,"log_dups_size":0,"ondisk_log_size":6360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26025984,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4370,"num_write_kb":17480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6403","reported_seq":6525,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.351601+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:12.351601+0000","last_peered":"2024-09-05T13:14:12.351601+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:12.351601+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6403,"log_dups_size":0,"ondisk_log_size":6403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26210304,"num_objects":6399,"num_object_clones":0,"num_object_copies":12798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4214,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6433","reported_seq":6497,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:12.387379+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:12.387379+0000","last_peered":"2024-09-05T13:14:12.387379+0000","last_clean":"2024-09-05T13:14:12.387379+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:12.387379+0000","last_undegraded":"2024-09-05T13:14:12.387379+0000","last_fullsized":"2024-09-05T13:14:12.387379+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6433,"log_dups_size":0,"ondisk_log_size":6433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26349568,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6371","reported_seq":6500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:11.514254+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:11.514254+0000","last_peered":"2024-09-05T13:14:11.514254+0000","last_clean":"2024-09-05T13:14:11.514254+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:11.514254+0000","last_undegraded":"2024-09-05T13:14:11.514254+0000","last_fullsized":"2024-09-05T13:14:11.514254+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6371,"log_dups_size":0,"ondisk_log_size":6371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2911,"num_write_kb":11644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6374","reported_seq":6507,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.491964+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:11.491964+0000","last_peered":"2024-09-05T13:14:11.491964+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:11.491964+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6374,"log_dups_size":0,"ondisk_log_size":6374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2829,"num_write_kb":11316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1918},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6283","reported_seq":6417,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.519098+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:11.519098+0000","last_peered":"2024-09-05T13:14:11.519098+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:11.519098+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710592,"num_objects":6277,"num_object_clones":0,"num_object_copies":12554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2602,"num_write_kb":10408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1841},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6432","reported_seq":6559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:11.537072+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:11.537072+0000","last_peered":"2024-09-05T13:14:11.537072+0000","last_clean":"2024-09-05T13:14:11.537072+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:11.537072+0000","last_undegraded":"2024-09-05T13:14:11.537072+0000","last_fullsized":"2024-09-05T13:14:11.537072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6432,"log_dups_size":0,"ondisk_log_size":6432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337280,"num_objects":6430,"num_object_clones":0,"num_object_copies":12860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6375","reported_seq":6504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.372604+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:12.372604+0000","last_peered":"2024-09-05T13:14:12.372604+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:12.372604+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6375,"log_dups_size":0,"ondisk_log_size":6375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26083328,"num_objects":6368,"num_object_clones":0,"num_object_copies":12736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4029,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6320","reported_seq":6453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.502784+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:11.502784+0000","last_peered":"2024-09-05T13:14:11.502784+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:11.502784+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25853952,"num_objects":6312,"num_object_clones":0,"num_object_copies":12624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1858},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6297","reported_seq":8828,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.387973+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:12.387973+0000","last_peered":"2024-09-05T13:14:12.387973+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:12.387973+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6297,"log_dups_size":0,"ondisk_log_size":6297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25772032,"num_objects":6292,"num_object_clones":0,"num_object_copies":12584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":6292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3892,"num_write_kb":15568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":4956160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3175},{"shards":"2","objects":3115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6257","reported_seq":6374,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.330178+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:12.330178+0000","last_peered":"2024-09-05T13:14:12.330178+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:12.330178+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6257,"log_dups_size":0,"ondisk_log_size":6257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25604096,"num_objects":6251,"num_object_clones":0,"num_object_copies":12502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3766,"num_write_kb":15064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1864}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6318","reported_seq":6450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.503713+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:11.503713+0000","last_peered":"2024-09-05T13:14:11.503713+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:11.503713+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6318,"log_dups_size":0,"ondisk_log_size":6318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25870336,"num_objects":6316,"num_object_clones":0,"num_object_copies":12632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2469,"num_write_kb":9876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1897},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6332","reported_seq":6514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.526856+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:11.526856+0000","last_peered":"2024-09-05T13:14:11.526856+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:11.526856+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6332,"log_dups_size":0,"ondisk_log_size":6332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25903104,"num_objects":6324,"num_object_clones":0,"num_object_copies":12648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2231,"num_write_kb":8924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1880},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6410","reported_seq":6541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.570933+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:11.570933+0000","last_peered":"2024-09-05T13:14:11.570933+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:11.570933+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6410,"log_dups_size":0,"ondisk_log_size":6410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26238976,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2540,"num_write_kb":10160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1840},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6458","reported_seq":6592,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.577671+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:11.577671+0000","last_peered":"2024-09-05T13:14:11.577671+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:11.577671+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6458,"log_dups_size":0,"ondisk_log_size":6458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26403299,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3126,"num_write_kb":12504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1882},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":416268771,"num_objects":101629,"num_object_clones":0,"num_object_copies":203258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":101629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57428,"num_write_kb":229712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1213,"num_bytes_recovered":4968448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":833093632,"data_stored":833003307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101710,"ondisk_log_size":101710,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522972,"kb_used_data":346892,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93848868,"statfs":{"total":96636764160,"available":96101240832,"internally_reserved":0,"allocated":355217408,"data_stored":354972318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573428,"kb_used_data":375204,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93798412,"statfs":{"total":96636764160,"available":96049573888,"internally_reserved":0,"allocated":384208896,"data_stored":383959729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":202960554},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212520,"kb_used_data":90584,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94159320,"statfs":{"total":96636764160,"available":96419143680,"internally_reserved":0,"allocated":92758016,"data_stored":92429363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":124841647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,4,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384950272,"data_stored":384946659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92983296,"data_stored":92900197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355160064,"data_stored":355156451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:14.823 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:15.209 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 102 20 103418 103398 3.95883 1.89062 0.0738674 0.0197267 2024-09-05T13:14:15.365 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:15.365 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:15.390 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-05T13:14:13.538749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":416268790,"num_objects":101631,"num_object_clones":0,"num_object_copies":203262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":101631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57430,"num_write_kb":229714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1214,"num_bytes_recovered":4968467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101713,"ondisk_log_size":101713,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1308920,"kb_used_data":812680,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281806600,"statfs":{"total":289910292480,"available":288569958400,"internally_reserved":0,"allocated":832184320,"data_stored":831361410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":508087302},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,7,7,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26214400,"num_objects":6400,"num_object_clones":0,"num_object_copies":12800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-355,"num_objects_unfound":0,"num_objects_dirty":6400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6400,"num_write_kb":25600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004130"},"pg_stats":[{"pgid":"2.f","version":"43'6287","reported_seq":6352,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.381387+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:11.381387+0000","last_peered":"2024-09-05T13:14:11.381387+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:11.381387+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25751552,"num_objects":6287,"num_object_clones":0,"num_object_copies":12574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1866},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6360","reported_seq":6458,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:12.386641+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:12.386641+0000","last_peered":"2024-09-05T13:14:12.386641+0000","last_clean":"2024-09-05T13:14:12.386641+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:12.386641+0000","last_undegraded":"2024-09-05T13:14:12.386641+0000","last_fullsized":"2024-09-05T13:14:12.386641+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6360,"log_dups_size":0,"ondisk_log_size":6360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26025984,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4370,"num_write_kb":17480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6403","reported_seq":6525,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.351601+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:12.351601+0000","last_peered":"2024-09-05T13:14:12.351601+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:12.351601+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6403,"log_dups_size":0,"ondisk_log_size":6403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26210304,"num_objects":6399,"num_object_clones":0,"num_object_copies":12798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4214,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6433","reported_seq":6497,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:12.387379+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:12.387379+0000","last_peered":"2024-09-05T13:14:12.387379+0000","last_clean":"2024-09-05T13:14:12.387379+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:12.387379+0000","last_undegraded":"2024-09-05T13:14:12.387379+0000","last_fullsized":"2024-09-05T13:14:12.387379+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6433,"log_dups_size":0,"ondisk_log_size":6433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26349568,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6371","reported_seq":6500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:11.514254+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:11.514254+0000","last_peered":"2024-09-05T13:14:11.514254+0000","last_clean":"2024-09-05T13:14:11.514254+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:11.514254+0000","last_undegraded":"2024-09-05T13:14:11.514254+0000","last_fullsized":"2024-09-05T13:14:11.514254+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6371,"log_dups_size":0,"ondisk_log_size":6371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2911,"num_write_kb":11644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6374","reported_seq":6507,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.491964+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:11.491964+0000","last_peered":"2024-09-05T13:14:11.491964+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:11.491964+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6374,"log_dups_size":0,"ondisk_log_size":6374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2829,"num_write_kb":11316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1918},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6283","reported_seq":6417,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.519098+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:11.519098+0000","last_peered":"2024-09-05T13:14:11.519098+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:11.519098+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710592,"num_objects":6277,"num_object_clones":0,"num_object_copies":12554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2602,"num_write_kb":10408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1841},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6432","reported_seq":6559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:11.537072+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:11.537072+0000","last_peered":"2024-09-05T13:14:11.537072+0000","last_clean":"2024-09-05T13:14:11.537072+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:11.537072+0000","last_undegraded":"2024-09-05T13:14:11.537072+0000","last_fullsized":"2024-09-05T13:14:11.537072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6432,"log_dups_size":0,"ondisk_log_size":6432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337280,"num_objects":6430,"num_object_clones":0,"num_object_copies":12860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6375","reported_seq":6504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.372604+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:12.372604+0000","last_peered":"2024-09-05T13:14:12.372604+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:12.372604+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6375,"log_dups_size":0,"ondisk_log_size":6375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26083328,"num_objects":6368,"num_object_clones":0,"num_object_copies":12736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4029,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6320","reported_seq":6453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.502784+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:11.502784+0000","last_peered":"2024-09-05T13:14:11.502784+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:11.502784+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25853952,"num_objects":6312,"num_object_clones":0,"num_object_copies":12624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1858},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6297","reported_seq":8828,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.387973+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:12.387973+0000","last_peered":"2024-09-05T13:14:12.387973+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:12.387973+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6297,"log_dups_size":0,"ondisk_log_size":6297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25772032,"num_objects":6292,"num_object_clones":0,"num_object_copies":12584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":6292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3892,"num_write_kb":15568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":4956160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3175},{"shards":"2","objects":3115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6257","reported_seq":6374,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.330178+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:12.330178+0000","last_peered":"2024-09-05T13:14:12.330178+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:12.330178+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6257,"log_dups_size":0,"ondisk_log_size":6257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25604096,"num_objects":6251,"num_object_clones":0,"num_object_copies":12502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3766,"num_write_kb":15064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1864}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6318","reported_seq":6450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.503713+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:11.503713+0000","last_peered":"2024-09-05T13:14:11.503713+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:11.503713+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6318,"log_dups_size":0,"ondisk_log_size":6318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25870336,"num_objects":6316,"num_object_clones":0,"num_object_copies":12632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2469,"num_write_kb":9876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1897},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6332","reported_seq":6514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.526856+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:11.526856+0000","last_peered":"2024-09-05T13:14:11.526856+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:11.526856+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6332,"log_dups_size":0,"ondisk_log_size":6332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25903104,"num_objects":6324,"num_object_clones":0,"num_object_copies":12648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2231,"num_write_kb":8924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1880},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6410","reported_seq":6541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.570933+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:11.570933+0000","last_peered":"2024-09-05T13:14:11.570933+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:11.570933+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6410,"log_dups_size":0,"ondisk_log_size":6410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26238976,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2540,"num_write_kb":10160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1840},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6458","reported_seq":6592,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.577671+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:11.577671+0000","last_peered":"2024-09-05T13:14:11.577671+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:11.577671+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6458,"log_dups_size":0,"ondisk_log_size":6458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26403299,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3126,"num_write_kb":12504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1882},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":416268771,"num_objects":101629,"num_object_clones":0,"num_object_copies":203258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":101629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57428,"num_write_kb":229712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1213,"num_bytes_recovered":4968448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":833093632,"data_stored":833003307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101710,"ondisk_log_size":101710,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522972,"kb_used_data":346892,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93848868,"statfs":{"total":96636764160,"available":96101240832,"internally_reserved":0,"allocated":355217408,"data_stored":354972318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573428,"kb_used_data":375204,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93798412,"statfs":{"total":96636764160,"available":96049573888,"internally_reserved":0,"allocated":384208896,"data_stored":383959729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":202960554},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":212520,"kb_used_data":90584,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94159320,"statfs":{"total":96636764160,"available":96419143680,"internally_reserved":0,"allocated":92758016,"data_stored":92429363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":124841647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,4,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384950272,"data_stored":384946659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":92983296,"data_stored":92900197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355160064,"data_stored":355156451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:15.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:15.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6287", 'reported_seq': 6352, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:11.381387+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:11.381387+0000', 'last_peered': '2024-09-05T13:14:11.381387+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:11.381387+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 6287, 'log_dups_size': 0, 'ondisk_log_size': 6287, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25751552, 'num_objects': 6287, 'num_object_clones': 0, 'num_object_copies': 12574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6287, 'num_write_kb': 25148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1866}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:15.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:15.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6403", 'reported_seq': 6525, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:12.351601+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:12.351601+0000', 'last_peered': '2024-09-05T13:14:12.351601+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:12.351601+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 6403, 'log_dups_size': 0, 'ondisk_log_size': 6403, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26210304, 'num_objects': 6399, 'num_object_clones': 0, 'num_object_copies': 12798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6399, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4214, 'num_write_kb': 16856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 1907}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:15.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:15.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6374", 'reported_seq': 6507, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:11.491964+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:11.491964+0000', 'last_peered': '2024-09-05T13:14:11.491964+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:11.491964+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 6374, 'log_dups_size': 0, 'ondisk_log_size': 6374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26087424, 'num_objects': 6369, 'num_object_clones': 0, 'num_object_copies': 12738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2829, 'num_write_kb': 11316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1918}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:15.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:15.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6283", 'reported_seq': 6417, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:11.519098+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:11.519098+0000', 'last_peered': '2024-09-05T13:14:11.519098+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:11.519098+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 6283, 'log_dups_size': 0, 'ondisk_log_size': 6283, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25710592, 'num_objects': 6277, 'num_object_clones': 0, 'num_object_copies': 12554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2602, 'num_write_kb': 10408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1841}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6375", 'reported_seq': 6504, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:12.372604+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:12.372604+0000', 'last_peered': '2024-09-05T13:14:12.372604+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:12.372604+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 6375, 'log_dups_size': 0, 'ondisk_log_size': 6375, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26083328, 'num_objects': 6368, 'num_object_clones': 0, 'num_object_copies': 12736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6368, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4029, 'num_write_kb': 16116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 1917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6320", 'reported_seq': 6453, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:11.502784+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:11.502784+0000', 'last_peered': '2024-09-05T13:14:11.502784+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:11.502784+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 6320, 'log_dups_size': 0, 'ondisk_log_size': 6320, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25853952, 'num_objects': 6312, 'num_object_clones': 0, 'num_object_copies': 12624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2687, 'num_write_kb': 10748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1858}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6297", 'reported_seq': 8828, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:12.387973+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:12.387973+0000', 'last_peered': '2024-09-05T13:14:12.387973+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:12.387973+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 6297, 'log_dups_size': 0, 'ondisk_log_size': 6297, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25772032, 'num_objects': 6292, 'num_object_clones': 0, 'num_object_copies': 12584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3175, 'num_objects_unfound': 0, 'num_objects_dirty': 6292, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3892, 'num_write_kb': 15568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1210, 'num_bytes_recovered': 4956160, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3175}, {'shards': '2', 'objects': 3115}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6257", 'reported_seq': 6374, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:12.330178+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:12.330178+0000', 'last_peered': '2024-09-05T13:14:12.330178+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:12.330178+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 6257, 'log_dups_size': 0, 'ondisk_log_size': 6257, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25604096, 'num_objects': 6251, 'num_object_clones': 0, 'num_object_copies': 12502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6251, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3766, 'num_write_kb': 15064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 1864}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6318", 'reported_seq': 6450, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:11.503713+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:11.503713+0000', 'last_peered': '2024-09-05T13:14:11.503713+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:11.503713+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 6318, 'log_dups_size': 0, 'ondisk_log_size': 6318, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25870336, 'num_objects': 6316, 'num_object_clones': 0, 'num_object_copies': 12632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2469, 'num_write_kb': 9876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1897}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:15.393 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:15.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6332", 'reported_seq': 6514, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:11.526856+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:11.526856+0000', 'last_peered': '2024-09-05T13:14:11.526856+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:11.526856+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 6332, 'log_dups_size': 0, 'ondisk_log_size': 6332, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25903104, 'num_objects': 6324, 'num_object_clones': 0, 'num_object_copies': 12648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2231, 'num_write_kb': 8924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1880}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:15.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:15.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6410", 'reported_seq': 6541, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:11.570933+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:11.570933+0000', 'last_peered': '2024-09-05T13:14:11.570933+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:11.570933+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 6410, 'log_dups_size': 0, 'ondisk_log_size': 6410, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26238976, 'num_objects': 6406, 'num_object_clones': 0, 'num_object_copies': 12812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2540, 'num_write_kb': 10160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1840}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:15.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:15.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6458", 'reported_seq': 6592, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:11.577671+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:11.577671+0000', 'last_peered': '2024-09-05T13:14:11.577671+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:11.577671+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 6458, 'log_dups_size': 0, 'ondisk_log_size': 6458, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26403299, 'num_objects': 6447, 'num_object_clones': 0, 'num_object_copies': 12894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3126, 'num_write_kb': 12504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1882}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:15.395 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:16.088 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:16.088 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/203262 objects degraded (24.160%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":182,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":101631,"data_bytes":416268790,"bytes_used":1340334080,"bytes_avail":288569958400,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":203262,"degraded_ratio":0.24159951195993348,"misplaced_objects":3175,"misplaced_total":203262,"misplaced_ratio":0.015620233983725439,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":121131,"recovering_keys_per_sec":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"write_bytes_sec":2183781,"read_op_per_sec":0,"write_op_per_sec":533},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-05T13:13:17.519932+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-05T13:14:16.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:16.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:16.210 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 103 20 103915 103895 3.93925 1.94141 0.0125042 0.0198237 2024-09-05T13:14:16.662 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:16.690 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-05T13:14:15.539525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":416268790,"num_objects":101631,"num_object_clones":0,"num_object_copies":203262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":101631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57430,"num_write_kb":229714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1214,"num_bytes_recovered":4968467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101713,"ondisk_log_size":101713,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1325672,"kb_used_data":821112,"kb_used_omap":12,"kb_used_meta":504499,"kb_avail":281789848,"statfs":{"total":289910292480,"available":288552804352,"internally_reserved":0,"allocated":840818688,"data_stored":839995778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":516606982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,5,3,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-239,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5370,"num_write_kb":21480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004126"},"pg_stats":[{"pgid":"2.f","version":"43'6287","reported_seq":6352,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.381387+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:11.381387+0000","last_peered":"2024-09-05T13:14:11.381387+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:11.381387+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25751552,"num_objects":6287,"num_object_clones":0,"num_object_copies":12574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1866},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6360","reported_seq":6458,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:12.386641+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:12.386641+0000","last_peered":"2024-09-05T13:14:12.386641+0000","last_clean":"2024-09-05T13:14:12.386641+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:12.386641+0000","last_undegraded":"2024-09-05T13:14:12.386641+0000","last_fullsized":"2024-09-05T13:14:12.386641+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6360,"log_dups_size":0,"ondisk_log_size":6360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26025984,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4370,"num_write_kb":17480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6403","reported_seq":6525,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.351601+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:12.351601+0000","last_peered":"2024-09-05T13:14:12.351601+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:12.351601+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6403,"log_dups_size":0,"ondisk_log_size":6403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26210304,"num_objects":6399,"num_object_clones":0,"num_object_copies":12798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4214,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":1907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6433","reported_seq":6497,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:12.387379+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:12.387379+0000","last_peered":"2024-09-05T13:14:12.387379+0000","last_clean":"2024-09-05T13:14:12.387379+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:12.387379+0000","last_undegraded":"2024-09-05T13:14:12.387379+0000","last_fullsized":"2024-09-05T13:14:12.387379+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6433,"log_dups_size":0,"ondisk_log_size":6433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26349568,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6371","reported_seq":6500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:11.514254+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:11.514254+0000","last_peered":"2024-09-05T13:14:11.514254+0000","last_clean":"2024-09-05T13:14:11.514254+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:11.514254+0000","last_undegraded":"2024-09-05T13:14:11.514254+0000","last_fullsized":"2024-09-05T13:14:11.514254+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6371,"log_dups_size":0,"ondisk_log_size":6371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2911,"num_write_kb":11644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6374","reported_seq":6507,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.491964+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:11.491964+0000","last_peered":"2024-09-05T13:14:11.491964+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:11.491964+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6374,"log_dups_size":0,"ondisk_log_size":6374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26087424,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2829,"num_write_kb":11316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1918},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6283","reported_seq":6417,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.519098+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:11.519098+0000","last_peered":"2024-09-05T13:14:11.519098+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:11.519098+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710592,"num_objects":6277,"num_object_clones":0,"num_object_copies":12554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2602,"num_write_kb":10408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1841},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6432","reported_seq":6559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:11.537072+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:11.537072+0000","last_peered":"2024-09-05T13:14:11.537072+0000","last_clean":"2024-09-05T13:14:11.537072+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:11.537072+0000","last_undegraded":"2024-09-05T13:14:11.537072+0000","last_fullsized":"2024-09-05T13:14:11.537072+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6432,"log_dups_size":0,"ondisk_log_size":6432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337280,"num_objects":6430,"num_object_clones":0,"num_object_copies":12860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3042,"num_write_kb":12168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6375","reported_seq":6504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.372604+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:12.372604+0000","last_peered":"2024-09-05T13:14:12.372604+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:12.372604+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6375,"log_dups_size":0,"ondisk_log_size":6375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26083328,"num_objects":6368,"num_object_clones":0,"num_object_copies":12736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4029,"num_write_kb":16116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6320","reported_seq":6453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.502784+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:11.502784+0000","last_peered":"2024-09-05T13:14:11.502784+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:11.502784+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25853952,"num_objects":6312,"num_object_clones":0,"num_object_copies":12624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1858},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6297","reported_seq":8828,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.387973+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:12.387973+0000","last_peered":"2024-09-05T13:14:12.387973+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:12.387973+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6297,"log_dups_size":0,"ondisk_log_size":6297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25772032,"num_objects":6292,"num_object_clones":0,"num_object_copies":12584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":6292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3892,"num_write_kb":15568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1210,"num_bytes_recovered":4956160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3175},{"shards":"2","objects":3115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6257","reported_seq":6374,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:12.330178+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:12.330178+0000","last_peered":"2024-09-05T13:14:12.330178+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:12.330178+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6257,"log_dups_size":0,"ondisk_log_size":6257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25604096,"num_objects":6251,"num_object_clones":0,"num_object_copies":12502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3766,"num_write_kb":15064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":1864}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6318","reported_seq":6450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.503713+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:11.503713+0000","last_peered":"2024-09-05T13:14:11.503713+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:11.503713+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6318,"log_dups_size":0,"ondisk_log_size":6318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25870336,"num_objects":6316,"num_object_clones":0,"num_object_copies":12632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2469,"num_write_kb":9876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1897},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6332","reported_seq":6514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.526856+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:11.526856+0000","last_peered":"2024-09-05T13:14:11.526856+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:11.526856+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6332,"log_dups_size":0,"ondisk_log_size":6332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25903104,"num_objects":6324,"num_object_clones":0,"num_object_copies":12648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2231,"num_write_kb":8924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1880},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6410","reported_seq":6541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.570933+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:11.570933+0000","last_peered":"2024-09-05T13:14:11.570933+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:11.570933+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6410,"log_dups_size":0,"ondisk_log_size":6410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26238976,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2540,"num_write_kb":10160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1840},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6458","reported_seq":6592,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:11.577671+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:11.577671+0000","last_peered":"2024-09-05T13:14:11.577671+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:11.577671+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6458,"log_dups_size":0,"ondisk_log_size":6458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26403299,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3126,"num_write_kb":12504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1882},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":416268771,"num_objects":101629,"num_object_clones":0,"num_object_copies":203258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3175,"num_objects_unfound":0,"num_objects_dirty":101629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57428,"num_write_kb":229712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1213,"num_bytes_recovered":4968448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841904128,"data_stored":841813803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101710,"ondisk_log_size":101710,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522972,"kb_used_data":346892,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93848868,"statfs":{"total":96636764160,"available":96101240832,"internally_reserved":0,"allocated":355217408,"data_stored":354972318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":180285101},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573428,"kb_used_data":375204,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93798412,"statfs":{"total":96636764160,"available":96049573888,"internally_reserved":0,"allocated":384208896,"data_stored":383959729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":202960554},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":229272,"kb_used_data":99016,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94142568,"statfs":{"total":96636764160,"available":96401989632,"internally_reserved":0,"allocated":101392384,"data_stored":101063731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133361327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,3,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384950272,"data_stored":384946659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":101793792,"data_stored":101710693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355160064,"data_stored":355156451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:17.210 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 104 20 104439 104419 3.92105 2.04688 0.0279097 0.0199173 2024-09-05T13:14:18.211 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 105 20 104992 104972 3.90424 2.16016 0.0157522 0.0200018 2024-09-05T13:14:19.214 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 106 20 105534 105514 3.88739 2.11719 0.0183842 0.0200897 2024-09-05T13:14:19.693 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:20.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 107 20 106056 106036 3.86998 2.03906 0.0273 0.0201806 2024-09-05T13:14:20.231 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:20.232 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:20.257 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-05T13:14:19.541039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":427061750,"num_objects":104266,"num_object_clones":0,"num_object_copies":208532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":104266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60065,"num_write_kb":240254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1323,"num_bytes_recovered":5414931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104348,"ondisk_log_size":104348,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1365024,"kb_used_data":842416,"kb_used_omap":12,"kb_used_meta":522547,"kb_avail":281750496,"statfs":{"total":289910292480,"available":288512507904,"internally_reserved":0,"allocated":862633984,"data_stored":861811074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":535088134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,5,11,8,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21725184,"num_objects":5304,"num_object_clones":0,"num_object_copies":10608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":5304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5304,"num_write_kb":21216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004603"},"pg_stats":[{"pgid":"2.f","version":"43'6451","reported_seq":6521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.600389+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:16.600389+0000","last_peered":"2024-09-05T13:14:16.600389+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:16.600389+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6451,"log_dups_size":0,"ondisk_log_size":6451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26423296,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6451,"num_write_kb":25804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2030},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6522","reported_seq":6621,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:17.415563+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:17.415563+0000","last_peered":"2024-09-05T13:14:17.415563+0000","last_clean":"2024-09-05T13:14:17.415563+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:17.415563+0000","last_undegraded":"2024-09-05T13:14:17.415563+0000","last_fullsized":"2024-09-05T13:14:17.415563+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6522,"log_dups_size":0,"ondisk_log_size":6522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4532,"num_write_kb":18128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6581","reported_seq":6703,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.373914+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:17.373914+0000","last_peered":"2024-09-05T13:14:17.373914+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:17.373914+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4392,"num_write_kb":17568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6587","reported_seq":6651,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:17.305686+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:17.305686+0000","last_peered":"2024-09-05T13:14:17.305686+0000","last_clean":"2024-09-05T13:14:17.305686+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:17.305686+0000","last_undegraded":"2024-09-05T13:14:17.305686+0000","last_fullsized":"2024-09-05T13:14:17.305686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6587,"log_dups_size":0,"ondisk_log_size":6587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6587,"num_write_kb":26348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6530","reported_seq":6659,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:16.568660+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:16.568660+0000","last_peered":"2024-09-05T13:14:16.568660+0000","last_clean":"2024-09-05T13:14:16.568660+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:16.568660+0000","last_undegraded":"2024-09-05T13:14:16.568660+0000","last_fullsized":"2024-09-05T13:14:16.568660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6530,"log_dups_size":0,"ondisk_log_size":6530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26718208,"num_objects":6523,"num_object_clones":0,"num_object_copies":13046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":12280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6519","reported_seq":6652,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.516525+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:16.516525+0000","last_peered":"2024-09-05T13:14:16.516525+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:16.516525+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6519,"log_dups_size":0,"ondisk_log_size":6519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26681344,"num_objects":6514,"num_object_clones":0,"num_object_copies":13028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6467","reported_seq":6601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.551830+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:16.551830+0000","last_peered":"2024-09-05T13:14:16.551830+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:16.551830+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26464256,"num_objects":6461,"num_object_clones":0,"num_object_copies":12922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2786,"num_write_kb":11144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2025},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6573","reported_seq":6700,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:16.461225+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:16.461225+0000","last_peered":"2024-09-05T13:14:16.461225+0000","last_clean":"2024-09-05T13:14:16.461225+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:16.461225+0000","last_undegraded":"2024-09-05T13:14:16.461225+0000","last_fullsized":"2024-09-05T13:14:16.461225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6573,"log_dups_size":0,"ondisk_log_size":6573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26914816,"num_objects":6571,"num_object_clones":0,"num_object_copies":13142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3183,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6544","reported_seq":6673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.364702+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:17.364702+0000","last_peered":"2024-09-05T13:14:17.364702+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:17.364702+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26775552,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6478","reported_seq":6611,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.520414+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:16.520414+0000","last_peered":"2024-09-05T13:14:16.520414+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:16.520414+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":11380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2016},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6465","reported_seq":9213,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.417453+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:17.417453+0000","last_peered":"2024-09-05T13:14:17.417453+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:17.417453+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6465,"log_dups_size":0,"ondisk_log_size":6465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26460160,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4060,"num_write_kb":16240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1319,"num_bytes_recovered":5402624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3066},{"shards":"2","objects":3392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6425","reported_seq":6543,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.403352+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:17.403352+0000","last_peered":"2024-09-05T13:14:17.403352+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:17.403352+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6425,"log_dups_size":0,"ondisk_log_size":6425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3934,"num_write_kb":15736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2032}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6469","reported_seq":6603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.574602+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:16.574602+0000","last_peered":"2024-09-05T13:14:16.574602+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:16.574602+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6469,"log_dups_size":0,"ondisk_log_size":6469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2620,"num_write_kb":10480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2048},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6514","reported_seq":6696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.531063+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:16.531063+0000","last_peered":"2024-09-05T13:14:16.531063+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:16.531063+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26648576,"num_objects":6506,"num_object_clones":0,"num_object_copies":13012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2413,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2062},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6581","reported_seq":6714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.585693+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:16.585693+0000","last_peered":"2024-09-05T13:14:16.585693+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:16.585693+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2711,"num_write_kb":10844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2011},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6639","reported_seq":6773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.467647+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:16.467647+0000","last_peered":"2024-09-05T13:14:16.467647+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:16.467647+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6639,"log_dups_size":0,"ondisk_log_size":6639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27144675,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3307,"num_write_kb":13228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":427061731,"num_objects":104264,"num_object_clones":0,"num_object_copies":208528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":104264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60063,"num_write_kb":240252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1322,"num_bytes_recovered":5414912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":863870976,"data_stored":863780651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104345,"ondisk_log_size":104345,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532364,"kb_used_data":352124,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93839476,"statfs":{"total":96636764160,"available":96091623424,"internally_reserved":0,"allocated":360574976,"data_stored":360329886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,6,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607532,"kb_used_data":382940,"kb_used_omap":4,"kb_used_meta":224571,"kb_avail":93764308,"statfs":{"total":96636764160,"available":96014651392,"internally_reserved":0,"allocated":392130560,"data_stored":391881393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":229961386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225128,"kb_used_data":107352,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94146712,"statfs":{"total":96636764160,"available":96406233088,"internally_reserved":0,"allocated":109928448,"data_stored":109599795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":120581807},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":392949760,"data_stored":392946147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110497792,"data_stored":110414693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360423424,"data_stored":360419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:20.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:20.843 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:20.844 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:20.872 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-05T13:14:19.541039+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":427061750,"num_objects":104266,"num_object_clones":0,"num_object_copies":208532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":104266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60065,"num_write_kb":240254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1323,"num_bytes_recovered":5414931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104348,"ondisk_log_size":104348,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1365024,"kb_used_data":842416,"kb_used_omap":12,"kb_used_meta":522547,"kb_avail":281750496,"statfs":{"total":289910292480,"available":288512507904,"internally_reserved":0,"allocated":862633984,"data_stored":861811074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":535088134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,5,11,8,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21725184,"num_objects":5304,"num_object_clones":0,"num_object_copies":10608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":5304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5304,"num_write_kb":21216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004603"},"pg_stats":[{"pgid":"2.f","version":"43'6451","reported_seq":6521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.600389+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:16.600389+0000","last_peered":"2024-09-05T13:14:16.600389+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:16.600389+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6451,"log_dups_size":0,"ondisk_log_size":6451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26423296,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6451,"num_write_kb":25804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2030},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6522","reported_seq":6621,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:17.415563+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:17.415563+0000","last_peered":"2024-09-05T13:14:17.415563+0000","last_clean":"2024-09-05T13:14:17.415563+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:17.415563+0000","last_undegraded":"2024-09-05T13:14:17.415563+0000","last_fullsized":"2024-09-05T13:14:17.415563+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6522,"log_dups_size":0,"ondisk_log_size":6522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4532,"num_write_kb":18128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6581","reported_seq":6703,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.373914+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:17.373914+0000","last_peered":"2024-09-05T13:14:17.373914+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:17.373914+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4392,"num_write_kb":17568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6587","reported_seq":6651,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:17.305686+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:17.305686+0000","last_peered":"2024-09-05T13:14:17.305686+0000","last_clean":"2024-09-05T13:14:17.305686+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:17.305686+0000","last_undegraded":"2024-09-05T13:14:17.305686+0000","last_fullsized":"2024-09-05T13:14:17.305686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6587,"log_dups_size":0,"ondisk_log_size":6587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6587,"num_write_kb":26348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6530","reported_seq":6659,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:16.568660+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:16.568660+0000","last_peered":"2024-09-05T13:14:16.568660+0000","last_clean":"2024-09-05T13:14:16.568660+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:16.568660+0000","last_undegraded":"2024-09-05T13:14:16.568660+0000","last_fullsized":"2024-09-05T13:14:16.568660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6530,"log_dups_size":0,"ondisk_log_size":6530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26718208,"num_objects":6523,"num_object_clones":0,"num_object_copies":13046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":12280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6519","reported_seq":6652,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.516525+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:16.516525+0000","last_peered":"2024-09-05T13:14:16.516525+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:16.516525+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6519,"log_dups_size":0,"ondisk_log_size":6519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26681344,"num_objects":6514,"num_object_clones":0,"num_object_copies":13028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6467","reported_seq":6601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.551830+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:16.551830+0000","last_peered":"2024-09-05T13:14:16.551830+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:16.551830+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26464256,"num_objects":6461,"num_object_clones":0,"num_object_copies":12922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2786,"num_write_kb":11144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2025},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6573","reported_seq":6700,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:16.461225+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:16.461225+0000","last_peered":"2024-09-05T13:14:16.461225+0000","last_clean":"2024-09-05T13:14:16.461225+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:16.461225+0000","last_undegraded":"2024-09-05T13:14:16.461225+0000","last_fullsized":"2024-09-05T13:14:16.461225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6573,"log_dups_size":0,"ondisk_log_size":6573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26914816,"num_objects":6571,"num_object_clones":0,"num_object_copies":13142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3183,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6544","reported_seq":6673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.364702+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:17.364702+0000","last_peered":"2024-09-05T13:14:17.364702+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:17.364702+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26775552,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6478","reported_seq":6611,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.520414+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:16.520414+0000","last_peered":"2024-09-05T13:14:16.520414+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:16.520414+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":11380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2016},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6465","reported_seq":9213,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.417453+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:17.417453+0000","last_peered":"2024-09-05T13:14:17.417453+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:17.417453+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6465,"log_dups_size":0,"ondisk_log_size":6465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26460160,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4060,"num_write_kb":16240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1319,"num_bytes_recovered":5402624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3066},{"shards":"2","objects":3392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6425","reported_seq":6543,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.403352+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:17.403352+0000","last_peered":"2024-09-05T13:14:17.403352+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:17.403352+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6425,"log_dups_size":0,"ondisk_log_size":6425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3934,"num_write_kb":15736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2032}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6469","reported_seq":6603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.574602+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:16.574602+0000","last_peered":"2024-09-05T13:14:16.574602+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:16.574602+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6469,"log_dups_size":0,"ondisk_log_size":6469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2620,"num_write_kb":10480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2048},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6514","reported_seq":6696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.531063+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:16.531063+0000","last_peered":"2024-09-05T13:14:16.531063+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:16.531063+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26648576,"num_objects":6506,"num_object_clones":0,"num_object_copies":13012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2413,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2062},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6581","reported_seq":6714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.585693+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:16.585693+0000","last_peered":"2024-09-05T13:14:16.585693+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:16.585693+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2711,"num_write_kb":10844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2011},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6639","reported_seq":6773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.467647+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:16.467647+0000","last_peered":"2024-09-05T13:14:16.467647+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:16.467647+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6639,"log_dups_size":0,"ondisk_log_size":6639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27144675,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3307,"num_write_kb":13228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":427061731,"num_objects":104264,"num_object_clones":0,"num_object_copies":208528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":104264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60063,"num_write_kb":240252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1322,"num_bytes_recovered":5414912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":863870976,"data_stored":863780651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104345,"ondisk_log_size":104345,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532364,"kb_used_data":352124,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93839476,"statfs":{"total":96636764160,"available":96091623424,"internally_reserved":0,"allocated":360574976,"data_stored":360329886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,6,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607532,"kb_used_data":382940,"kb_used_omap":4,"kb_used_meta":224571,"kb_avail":93764308,"statfs":{"total":96636764160,"available":96014651392,"internally_reserved":0,"allocated":392130560,"data_stored":391881393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":229961386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225128,"kb_used_data":107352,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94146712,"statfs":{"total":96636764160,"available":96406233088,"internally_reserved":0,"allocated":109928448,"data_stored":109599795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":120581807},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":392949760,"data_stored":392946147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110497792,"data_stored":110414693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360423424,"data_stored":360419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:20.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:20.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6451", 'reported_seq': 6521, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:16.600389+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:16.600389+0000', 'last_peered': '2024-09-05T13:14:16.600389+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:16.600389+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 6451, 'log_dups_size': 0, 'ondisk_log_size': 6451, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26423296, 'num_objects': 6451, 'num_object_clones': 0, 'num_object_copies': 12902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6451, 'num_write_kb': 25804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2030}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6581", 'reported_seq': 6703, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:17.373914+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:17.373914+0000', 'last_peered': '2024-09-05T13:14:17.373914+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:17.373914+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 6581, 'log_dups_size': 0, 'ondisk_log_size': 6581, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26939392, 'num_objects': 6577, 'num_object_clones': 0, 'num_object_copies': 13154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4392, 'num_write_kb': 17568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 2085}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6519", 'reported_seq': 6652, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:16.516525+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:16.516525+0000', 'last_peered': '2024-09-05T13:14:16.516525+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:16.516525+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 6519, 'log_dups_size': 0, 'ondisk_log_size': 6519, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26681344, 'num_objects': 6514, 'num_object_clones': 0, 'num_object_copies': 13028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2974, 'num_write_kb': 11896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2063}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6467", 'reported_seq': 6601, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:16.551830+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:16.551830+0000', 'last_peered': '2024-09-05T13:14:16.551830+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:16.551830+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 6467, 'log_dups_size': 0, 'ondisk_log_size': 6467, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26464256, 'num_objects': 6461, 'num_object_clones': 0, 'num_object_copies': 12922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6461, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2786, 'num_write_kb': 11144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2025}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6544", 'reported_seq': 6673, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:17.364702+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:17.364702+0000', 'last_peered': '2024-09-05T13:14:17.364702+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:17.364702+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 6544, 'log_dups_size': 0, 'ondisk_log_size': 6544, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26775552, 'num_objects': 6537, 'num_object_clones': 0, 'num_object_copies': 13074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6537, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4198, 'num_write_kb': 16792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 2086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:20.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6478", 'reported_seq': 6611, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:16.520414+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:16.520414+0000', 'last_peered': '2024-09-05T13:14:16.520414+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:16.520414+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 6478, 'log_dups_size': 0, 'ondisk_log_size': 6478, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26501120, 'num_objects': 6470, 'num_object_clones': 0, 'num_object_copies': 12940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6470, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2845, 'num_write_kb': 11380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2016}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6465", 'reported_seq': 9213, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:17.417453+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:17.417453+0000', 'last_peered': '2024-09-05T13:14:17.417453+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:17.417453+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 6465, 'log_dups_size': 0, 'ondisk_log_size': 6465, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26460160, 'num_objects': 6460, 'num_object_clones': 0, 'num_object_copies': 12920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3066, 'num_objects_unfound': 0, 'num_objects_dirty': 6460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4060, 'num_write_kb': 16240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1319, 'num_bytes_recovered': 5402624, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3066}, {'shards': '2', 'objects': 3392}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6425", 'reported_seq': 6543, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:17.403352+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:17.403352+0000', 'last_peered': '2024-09-05T13:14:17.403352+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:17.403352+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 6425, 'log_dups_size': 0, 'ondisk_log_size': 6425, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26292224, 'num_objects': 6419, 'num_object_clones': 0, 'num_object_copies': 12838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3934, 'num_write_kb': 15736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 2032}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6469", 'reported_seq': 6603, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:16.574602+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:16.574602+0000', 'last_peered': '2024-09-05T13:14:16.574602+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:16.574602+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 6469, 'log_dups_size': 0, 'ondisk_log_size': 6469, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26488832, 'num_objects': 6467, 'num_object_clones': 0, 'num_object_copies': 12934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2620, 'num_write_kb': 10480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2048}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6514", 'reported_seq': 6696, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:16.531063+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:16.531063+0000', 'last_peered': '2024-09-05T13:14:16.531063+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:16.531063+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 6514, 'log_dups_size': 0, 'ondisk_log_size': 6514, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26648576, 'num_objects': 6506, 'num_object_clones': 0, 'num_object_copies': 13012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6506, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2413, 'num_write_kb': 9652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2062}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6581", 'reported_seq': 6714, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:16.585693+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:16.585693+0000', 'last_peered': '2024-09-05T13:14:16.585693+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:16.585693+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 6581, 'log_dups_size': 0, 'ondisk_log_size': 6581, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26939392, 'num_objects': 6577, 'num_object_clones': 0, 'num_object_copies': 13154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2711, 'num_write_kb': 10844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2011}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:20.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6639", 'reported_seq': 6773, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:16.467647+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:16.467647+0000', 'last_peered': '2024-09-05T13:14:16.467647+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:16.467647+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 6639, 'log_dups_size': 0, 'ondisk_log_size': 6639, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27144675, 'num_objects': 6628, 'num_object_clones': 0, 'num_object_copies': 13256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6628, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3307, 'num_write_kb': 13228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2063}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:20.876 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:21.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 108 20 106546 106526 3.85187 1.91406 0.0326182 0.0202736 2024-09-05T13:14:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:21.561 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/208532 objects degraded (23.549%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":188,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":104266,"data_bytes":427061750,"bytes_used":1397784576,"bytes_avail":288512507904,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":208532,"degraded_ratio":0.23549383308077418,"misplaced_objects":3066,"misplaced_total":208532,"misplaced_ratio":0.014702779429535994,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":77794,"recovering_keys_per_sec":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"write_bytes_sec":1809737,"read_op_per_sec":0,"write_op_per_sec":441},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:14:21.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:21.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:22.155 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:22.181 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-05T13:14:21.541866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":427061750,"num_objects":104266,"num_object_clones":0,"num_object_copies":208532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":104266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60065,"num_write_kb":240254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1323,"num_bytes_recovered":5414931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104348,"ondisk_log_size":104348,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1365024,"kb_used_data":842416,"kb_used_omap":12,"kb_used_meta":522547,"kb_avail":281750496,"statfs":{"total":289910292480,"available":288512507904,"internally_reserved":0,"allocated":862633984,"data_stored":861811074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":535088134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,5,11,8,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21725184,"num_objects":5304,"num_object_clones":0,"num_object_copies":10608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":5304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5304,"num_write_kb":21216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004568"},"pg_stats":[{"pgid":"2.f","version":"43'6451","reported_seq":6521,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.600389+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:16.600389+0000","last_peered":"2024-09-05T13:14:16.600389+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:16.600389+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6451,"log_dups_size":0,"ondisk_log_size":6451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26423296,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6451,"num_write_kb":25804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2030},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6522","reported_seq":6621,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:17.415563+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:17.415563+0000","last_peered":"2024-09-05T13:14:17.415563+0000","last_clean":"2024-09-05T13:14:17.415563+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:17.415563+0000","last_undegraded":"2024-09-05T13:14:17.415563+0000","last_fullsized":"2024-09-05T13:14:17.415563+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6522,"log_dups_size":0,"ondisk_log_size":6522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4532,"num_write_kb":18128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6581","reported_seq":6703,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.373914+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:17.373914+0000","last_peered":"2024-09-05T13:14:17.373914+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:17.373914+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4392,"num_write_kb":17568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6587","reported_seq":6651,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:17.305686+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:17.305686+0000","last_peered":"2024-09-05T13:14:17.305686+0000","last_clean":"2024-09-05T13:14:17.305686+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:17.305686+0000","last_undegraded":"2024-09-05T13:14:17.305686+0000","last_fullsized":"2024-09-05T13:14:17.305686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6587,"log_dups_size":0,"ondisk_log_size":6587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6587,"num_write_kb":26348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6530","reported_seq":6659,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:16.568660+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:16.568660+0000","last_peered":"2024-09-05T13:14:16.568660+0000","last_clean":"2024-09-05T13:14:16.568660+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:16.568660+0000","last_undegraded":"2024-09-05T13:14:16.568660+0000","last_fullsized":"2024-09-05T13:14:16.568660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6530,"log_dups_size":0,"ondisk_log_size":6530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26718208,"num_objects":6523,"num_object_clones":0,"num_object_copies":13046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":12280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6519","reported_seq":6652,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.516525+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:16.516525+0000","last_peered":"2024-09-05T13:14:16.516525+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:16.516525+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6519,"log_dups_size":0,"ondisk_log_size":6519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26681344,"num_objects":6514,"num_object_clones":0,"num_object_copies":13028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2974,"num_write_kb":11896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6467","reported_seq":6601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.551830+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:16.551830+0000","last_peered":"2024-09-05T13:14:16.551830+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:16.551830+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26464256,"num_objects":6461,"num_object_clones":0,"num_object_copies":12922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2786,"num_write_kb":11144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2025},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6573","reported_seq":6700,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:16.461225+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:16.461225+0000","last_peered":"2024-09-05T13:14:16.461225+0000","last_clean":"2024-09-05T13:14:16.461225+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:16.461225+0000","last_undegraded":"2024-09-05T13:14:16.461225+0000","last_fullsized":"2024-09-05T13:14:16.461225+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6573,"log_dups_size":0,"ondisk_log_size":6573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26914816,"num_objects":6571,"num_object_clones":0,"num_object_copies":13142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3183,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6544","reported_seq":6673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.364702+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:17.364702+0000","last_peered":"2024-09-05T13:14:17.364702+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:17.364702+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26775552,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4198,"num_write_kb":16792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957056+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957056+0000","last_peered":"2024-09-05T13:13:17.957056+0000","last_clean":"2024-09-05T13:13:17.957056+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957056+0000","last_undegraded":"2024-09-05T13:13:17.957056+0000","last_fullsized":"2024-09-05T13:13:17.957056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6478","reported_seq":6611,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.520414+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:16.520414+0000","last_peered":"2024-09-05T13:14:16.520414+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:16.520414+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6478,"log_dups_size":0,"ondisk_log_size":6478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2845,"num_write_kb":11380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2016},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6465","reported_seq":9213,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.417453+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:17.417453+0000","last_peered":"2024-09-05T13:14:17.417453+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:17.417453+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6465,"log_dups_size":0,"ondisk_log_size":6465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26460160,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4060,"num_write_kb":16240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1319,"num_bytes_recovered":5402624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3066},{"shards":"2","objects":3392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6425","reported_seq":6543,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:17.403352+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:17.403352+0000","last_peered":"2024-09-05T13:14:17.403352+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:17.403352+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6425,"log_dups_size":0,"ondisk_log_size":6425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292224,"num_objects":6419,"num_object_clones":0,"num_object_copies":12838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3934,"num_write_kb":15736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2032}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6469","reported_seq":6603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.574602+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:16.574602+0000","last_peered":"2024-09-05T13:14:16.574602+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:16.574602+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6469,"log_dups_size":0,"ondisk_log_size":6469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2620,"num_write_kb":10480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2048},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6514","reported_seq":6696,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.531063+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:16.531063+0000","last_peered":"2024-09-05T13:14:16.531063+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:16.531063+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6514,"log_dups_size":0,"ondisk_log_size":6514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26648576,"num_objects":6506,"num_object_clones":0,"num_object_copies":13012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2413,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2062},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945415+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945415+0000","last_peered":"2024-09-05T13:13:17.945415+0000","last_clean":"2024-09-05T13:13:17.945415+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945415+0000","last_undegraded":"2024-09-05T13:13:17.945415+0000","last_fullsized":"2024-09-05T13:13:17.945415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6581","reported_seq":6714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.585693+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:16.585693+0000","last_peered":"2024-09-05T13:14:16.585693+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:16.585693+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6581,"log_dups_size":0,"ondisk_log_size":6581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2711,"num_write_kb":10844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2011},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6639","reported_seq":6773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:16.467647+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:16.467647+0000","last_peered":"2024-09-05T13:14:16.467647+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:16.467647+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6639,"log_dups_size":0,"ondisk_log_size":6639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27144675,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3307,"num_write_kb":13228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2063},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":427061731,"num_objects":104264,"num_object_clones":0,"num_object_copies":208528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":3066,"num_objects_unfound":0,"num_objects_dirty":104264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60063,"num_write_kb":240252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1322,"num_bytes_recovered":5414912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":863870976,"data_stored":863780651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104345,"ondisk_log_size":104345,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532364,"kb_used_data":352124,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93839476,"statfs":{"total":96636764160,"available":96091623424,"internally_reserved":0,"allocated":360574976,"data_stored":360329886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":184544941},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,6,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607532,"kb_used_data":382940,"kb_used_omap":4,"kb_used_meta":224571,"kb_avail":93764308,"statfs":{"total":96636764160,"available":96014651392,"internally_reserved":0,"allocated":392130560,"data_stored":391881393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":229961386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":225128,"kb_used_data":107352,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94146712,"statfs":{"total":96636764160,"available":96406233088,"internally_reserved":0,"allocated":109928448,"data_stored":109599795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":120581807},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":392949760,"data_stored":392946147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":110497792,"data_stored":110414693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360423424,"data_stored":360419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:22.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 109 20 107033 107013 3.83399 1.90234 0.0230606 0.0203701 2024-09-05T13:14:23.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 110 20 107589 107569 3.81888 2.17188 0.0149881 0.0204482 2024-09-05T13:14:24.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 111 20 108123 108103 3.80327 2.08594 0.0169086 0.0205336 2024-09-05T13:14:25.183 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:25.218 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 112 20 108663 108643 3.78805 2.10938 0.0169238 0.0206161 2024-09-05T13:14:25.720 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:25.723 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:25.743 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-05T13:14:25.544036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437715446,"num_objects":106867,"num_object_clones":0,"num_object_copies":213734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2949,"num_objects_unfound":0,"num_objects_dirty":106867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62666,"num_write_kb":250658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1440,"num_bytes_recovered":5894163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106949,"ondisk_log_size":106949,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1440332,"kb_used_data":863580,"kb_used_omap":12,"kb_used_meta":576691,"kb_avail":281675188,"statfs":{"total":289910292480,"available":288435392512,"internally_reserved":0,"allocated":884305920,"data_stored":883483010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":590531590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,6,11,10,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-226,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5236,"num_write_kb":20944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":925696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005287"},"pg_stats":[{"pgid":"2.f","version":"43'6626","reported_seq":6695,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.566176+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:21.566176+0000","last_peered":"2024-09-05T13:14:21.566176+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:21.566176+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6626,"log_dups_size":0,"ondisk_log_size":6626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27140096,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":26504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2205},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6659","reported_seq":6758,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:22.420979+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:22.420979+0000","last_peered":"2024-09-05T13:14:22.420979+0000","last_clean":"2024-09-05T13:14:22.420979+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:22.420979+0000","last_undegraded":"2024-09-05T13:14:22.420979+0000","last_fullsized":"2024-09-05T13:14:22.420979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6659,"log_dups_size":0,"ondisk_log_size":6659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27250688,"num_objects":6653,"num_object_clones":0,"num_object_copies":13306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6748","reported_seq":6870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:22.370127+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:22.370127+0000","last_peered":"2024-09-05T13:14:22.370127+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:22.370127+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4559,"num_write_kb":18236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2252}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6772","reported_seq":6836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:22.389820+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:22.389820+0000","last_peered":"2024-09-05T13:14:22.389820+0000","last_clean":"2024-09-05T13:14:22.389820+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:22.389820+0000","last_undegraded":"2024-09-05T13:14:22.389820+0000","last_fullsized":"2024-09-05T13:14:22.389820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6772,"log_dups_size":0,"ondisk_log_size":6772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6772,"num_write_kb":27088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6686","reported_seq":6815,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:21.443138+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:21.443138+0000","last_peered":"2024-09-05T13:14:21.443138+0000","last_clean":"2024-09-05T13:14:21.443138+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:21.443138+0000","last_undegraded":"2024-09-05T13:14:21.443138+0000","last_fullsized":"2024-09-05T13:14:21.443138+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6686,"log_dups_size":0,"ondisk_log_size":6686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3226,"num_write_kb":12904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6693","reported_seq":6823,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.549235+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:21.549235+0000","last_peered":"2024-09-05T13:14:21.549235+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:21.549235+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6693,"log_dups_size":0,"ondisk_log_size":6693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27394048,"num_objects":6688,"num_object_clones":0,"num_object_copies":13376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6634","reported_seq":6767,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.576447+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:21.576447+0000","last_peered":"2024-09-05T13:14:21.576447+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:21.576447+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6634,"log_dups_size":0,"ondisk_log_size":6634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2953,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2192},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6738","reported_seq":6865,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:21.558672+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:21.558672+0000","last_peered":"2024-09-05T13:14:21.558672+0000","last_clean":"2024-09-05T13:14:21.558672+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:21.558672+0000","last_undegraded":"2024-09-05T13:14:21.558672+0000","last_fullsized":"2024-09-05T13:14:21.558672+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6738,"log_dups_size":0,"ondisk_log_size":6738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27590656,"num_objects":6736,"num_object_clones":0,"num_object_copies":13472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3348,"num_write_kb":13392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6684","reported_seq":6813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:22.362911+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:22.362911+0000","last_peered":"2024-09-05T13:14:22.362911+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:22.362911+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6684,"log_dups_size":0,"ondisk_log_size":6684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27348992,"num_objects":6677,"num_object_clones":0,"num_object_copies":13354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4338,"num_write_kb":17352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6634","reported_seq":6767,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.482289+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:21.482289+0000","last_peered":"2024-09-05T13:14:21.482289+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:21.482289+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6634,"log_dups_size":0,"ondisk_log_size":6634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27140096,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3001,"num_write_kb":12004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2172},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6619","reported_seq":9601,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:22.380339+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:22.380339+0000","last_peered":"2024-09-05T13:14:22.380339+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:22.380339+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6619,"log_dups_size":0,"ondisk_log_size":6619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2949,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4214,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1436,"num_bytes_recovered":5881856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2949},{"shards":"2","objects":3663}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6586","reported_seq":6704,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:22.399342+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:22.399342+0000","last_peered":"2024-09-05T13:14:22.399342+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:22.399342+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6586,"log_dups_size":0,"ondisk_log_size":6586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26951680,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4095,"num_write_kb":16380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2193}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6633","reported_seq":6768,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.479081+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:21.479081+0000","last_peered":"2024-09-05T13:14:21.479081+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:21.479081+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6633,"log_dups_size":0,"ondisk_log_size":6633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27160576,"num_objects":6631,"num_object_clones":0,"num_object_copies":13262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2784,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2212},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6703","reported_seq":6885,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.583746+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:21.583746+0000","last_peered":"2024-09-05T13:14:21.583746+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:21.583746+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6703,"log_dups_size":0,"ondisk_log_size":6703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27422720,"num_objects":6695,"num_object_clones":0,"num_object_copies":13390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2602,"num_write_kb":10408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2251},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6734","reported_seq":6868,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.614066+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:21.614066+0000","last_peered":"2024-09-05T13:14:21.614066+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:21.614066+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6734,"log_dups_size":0,"ondisk_log_size":6734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2864,"num_write_kb":11456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2164},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6797","reported_seq":6932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.605003+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:21.605003+0000","last_peered":"2024-09-05T13:14:21.605003+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:21.605003+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6797,"log_dups_size":0,"ondisk_log_size":6797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27791843,"num_objects":6786,"num_object_clones":0,"num_object_copies":13572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3465,"num_write_kb":13860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2221},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437715427,"num_objects":106865,"num_object_clones":0,"num_object_copies":213730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2949,"num_objects_unfound":0,"num_objects_dirty":106865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62664,"num_write_kb":250656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1439,"num_bytes_recovered":5894144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":885714944,"data_stored":885624619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106946,"ondisk_log_size":106946,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541572,"kb_used_data":357172,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93830268,"statfs":{"total":96636764160,"available":96082194432,"internally_reserved":0,"allocated":365744128,"data_stored":365499038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657200,"kb_used_data":390944,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93714640,"statfs":{"total":96636764160,"available":95963791360,"internally_reserved":0,"allocated":400326656,"data_stored":400077489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241560,"kb_used_data":115464,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94130280,"statfs":{"total":96636764160,"available":96389406720,"internally_reserved":0,"allocated":118235136,"data_stored":117906483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129101487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401080320,"data_stored":401076707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":118964224,"data_stored":118881125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365670400,"data_stored":365666787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:25.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:26.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 113 20 109135 109115 3.77082 1.84375 0.106967 0.0207105 2024-09-05T13:14:26.350 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:26.351 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:26.371 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-05T13:14:25.544036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":437715446,"num_objects":106867,"num_object_clones":0,"num_object_copies":213734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2949,"num_objects_unfound":0,"num_objects_dirty":106867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62666,"num_write_kb":250658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1440,"num_bytes_recovered":5894163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106949,"ondisk_log_size":106949,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1440332,"kb_used_data":863580,"kb_used_omap":12,"kb_used_meta":576691,"kb_avail":281675188,"statfs":{"total":289910292480,"available":288435392512,"internally_reserved":0,"allocated":884305920,"data_stored":883483010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":590531590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,6,11,10,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-226,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5236,"num_write_kb":20944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":925696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005287"},"pg_stats":[{"pgid":"2.f","version":"43'6626","reported_seq":6695,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.566176+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:21.566176+0000","last_peered":"2024-09-05T13:14:21.566176+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:21.566176+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6626,"log_dups_size":0,"ondisk_log_size":6626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27140096,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":26504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2205},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6659","reported_seq":6758,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:22.420979+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:22.420979+0000","last_peered":"2024-09-05T13:14:22.420979+0000","last_clean":"2024-09-05T13:14:22.420979+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:22.420979+0000","last_undegraded":"2024-09-05T13:14:22.420979+0000","last_fullsized":"2024-09-05T13:14:22.420979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6659,"log_dups_size":0,"ondisk_log_size":6659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27250688,"num_objects":6653,"num_object_clones":0,"num_object_copies":13306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6748","reported_seq":6870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:22.370127+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:22.370127+0000","last_peered":"2024-09-05T13:14:22.370127+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:22.370127+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4559,"num_write_kb":18236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2252}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6772","reported_seq":6836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:22.389820+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:22.389820+0000","last_peered":"2024-09-05T13:14:22.389820+0000","last_clean":"2024-09-05T13:14:22.389820+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:22.389820+0000","last_undegraded":"2024-09-05T13:14:22.389820+0000","last_fullsized":"2024-09-05T13:14:22.389820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6772,"log_dups_size":0,"ondisk_log_size":6772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6772,"num_write_kb":27088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6686","reported_seq":6815,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:21.443138+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:21.443138+0000","last_peered":"2024-09-05T13:14:21.443138+0000","last_clean":"2024-09-05T13:14:21.443138+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:21.443138+0000","last_undegraded":"2024-09-05T13:14:21.443138+0000","last_fullsized":"2024-09-05T13:14:21.443138+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6686,"log_dups_size":0,"ondisk_log_size":6686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3226,"num_write_kb":12904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6693","reported_seq":6823,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.549235+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:21.549235+0000","last_peered":"2024-09-05T13:14:21.549235+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:21.549235+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6693,"log_dups_size":0,"ondisk_log_size":6693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27394048,"num_objects":6688,"num_object_clones":0,"num_object_copies":13376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6634","reported_seq":6767,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.576447+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:21.576447+0000","last_peered":"2024-09-05T13:14:21.576447+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:21.576447+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6634,"log_dups_size":0,"ondisk_log_size":6634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2953,"num_write_kb":11812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2192},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6738","reported_seq":6865,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:21.558672+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:21.558672+0000","last_peered":"2024-09-05T13:14:21.558672+0000","last_clean":"2024-09-05T13:14:21.558672+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:21.558672+0000","last_undegraded":"2024-09-05T13:14:21.558672+0000","last_fullsized":"2024-09-05T13:14:21.558672+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6738,"log_dups_size":0,"ondisk_log_size":6738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27590656,"num_objects":6736,"num_object_clones":0,"num_object_copies":13472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3348,"num_write_kb":13392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6684","reported_seq":6813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:22.362911+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:22.362911+0000","last_peered":"2024-09-05T13:14:22.362911+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:22.362911+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6684,"log_dups_size":0,"ondisk_log_size":6684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27348992,"num_objects":6677,"num_object_clones":0,"num_object_copies":13354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4338,"num_write_kb":17352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6634","reported_seq":6767,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.482289+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:21.482289+0000","last_peered":"2024-09-05T13:14:21.482289+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:21.482289+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6634,"log_dups_size":0,"ondisk_log_size":6634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27140096,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3001,"num_write_kb":12004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2172},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6619","reported_seq":9601,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:22.380339+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:22.380339+0000","last_peered":"2024-09-05T13:14:22.380339+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:22.380339+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6619,"log_dups_size":0,"ondisk_log_size":6619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2949,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4214,"num_write_kb":16856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1436,"num_bytes_recovered":5881856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2949},{"shards":"2","objects":3663}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6586","reported_seq":6704,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:22.399342+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:22.399342+0000","last_peered":"2024-09-05T13:14:22.399342+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:22.399342+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6586,"log_dups_size":0,"ondisk_log_size":6586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26951680,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4095,"num_write_kb":16380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2193}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6633","reported_seq":6768,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.479081+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:21.479081+0000","last_peered":"2024-09-05T13:14:21.479081+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:21.479081+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6633,"log_dups_size":0,"ondisk_log_size":6633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27160576,"num_objects":6631,"num_object_clones":0,"num_object_copies":13262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2784,"num_write_kb":11136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2212},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6703","reported_seq":6885,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.583746+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:21.583746+0000","last_peered":"2024-09-05T13:14:21.583746+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:21.583746+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6703,"log_dups_size":0,"ondisk_log_size":6703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27422720,"num_objects":6695,"num_object_clones":0,"num_object_copies":13390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2602,"num_write_kb":10408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2251},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6734","reported_seq":6868,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.614066+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:21.614066+0000","last_peered":"2024-09-05T13:14:21.614066+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:21.614066+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6734,"log_dups_size":0,"ondisk_log_size":6734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2864,"num_write_kb":11456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2164},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6797","reported_seq":6932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:21.605003+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:21.605003+0000","last_peered":"2024-09-05T13:14:21.605003+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:21.605003+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6797,"log_dups_size":0,"ondisk_log_size":6797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27791843,"num_objects":6786,"num_object_clones":0,"num_object_copies":13572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3465,"num_write_kb":13860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2221},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":437715427,"num_objects":106865,"num_object_clones":0,"num_object_copies":213730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2949,"num_objects_unfound":0,"num_objects_dirty":106865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62664,"num_write_kb":250656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1439,"num_bytes_recovered":5894144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":885714944,"data_stored":885624619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106946,"ondisk_log_size":106946,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541572,"kb_used_data":357172,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93830268,"statfs":{"total":96636764160,"available":96082194432,"internally_reserved":0,"allocated":365744128,"data_stored":365499038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":188804781},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657200,"kb_used_data":390944,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93714640,"statfs":{"total":96636764160,"available":95963791360,"internally_reserved":0,"allocated":400326656,"data_stored":400077489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241560,"kb_used_data":115464,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94130280,"statfs":{"total":96636764160,"available":96389406720,"internally_reserved":0,"allocated":118235136,"data_stored":117906483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129101487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401080320,"data_stored":401076707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":118964224,"data_stored":118881125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365670400,"data_stored":365666787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:26.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:26.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6626", 'reported_seq': 6695, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:21.566176+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:21.566176+0000', 'last_peered': '2024-09-05T13:14:21.566176+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:21.566176+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 6626, 'log_dups_size': 0, 'ondisk_log_size': 6626, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27140096, 'num_objects': 6626, 'num_object_clones': 0, 'num_object_copies': 13252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6626, 'num_write_kb': 26504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2205}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:26.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:26.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6748", 'reported_seq': 6870, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:22.370127+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:22.370127+0000', 'last_peered': '2024-09-05T13:14:22.370127+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:22.370127+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 6748, 'log_dups_size': 0, 'ondisk_log_size': 6748, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27623424, 'num_objects': 6744, 'num_object_clones': 0, 'num_object_copies': 13488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4559, 'num_write_kb': 18236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 2252}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:26.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:26.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6693", 'reported_seq': 6823, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:21.549235+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:21.549235+0000', 'last_peered': '2024-09-05T13:14:21.549235+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:21.549235+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 6693, 'log_dups_size': 0, 'ondisk_log_size': 6693, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27394048, 'num_objects': 6688, 'num_object_clones': 0, 'num_object_copies': 13376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6688, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3148, 'num_write_kb': 12592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2237}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:26.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6634", 'reported_seq': 6767, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:21.576447+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:21.576447+0000', 'last_peered': '2024-09-05T13:14:21.576447+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:21.576447+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 6634, 'log_dups_size': 0, 'ondisk_log_size': 6634, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27148288, 'num_objects': 6628, 'num_object_clones': 0, 'num_object_copies': 13256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6628, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2953, 'num_write_kb': 11812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2192}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6684", 'reported_seq': 6813, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:22.362911+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:22.362911+0000', 'last_peered': '2024-09-05T13:14:22.362911+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:22.362911+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 6684, 'log_dups_size': 0, 'ondisk_log_size': 6684, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27348992, 'num_objects': 6677, 'num_object_clones': 0, 'num_object_copies': 13354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6677, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4338, 'num_write_kb': 17352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 2226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6634", 'reported_seq': 6767, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:21.482289+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:21.482289+0000', 'last_peered': '2024-09-05T13:14:21.482289+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:21.482289+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 6634, 'log_dups_size': 0, 'ondisk_log_size': 6634, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27140096, 'num_objects': 6626, 'num_object_clones': 0, 'num_object_copies': 13252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3001, 'num_write_kb': 12004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2172}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6619", 'reported_seq': 9601, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:22.380339+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:22.380339+0000', 'last_peered': '2024-09-05T13:14:22.380339+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:22.380339+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 6619, 'log_dups_size': 0, 'ondisk_log_size': 6619, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27090944, 'num_objects': 6614, 'num_object_clones': 0, 'num_object_copies': 13228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2949, 'num_objects_unfound': 0, 'num_objects_dirty': 6614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4214, 'num_write_kb': 16856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1436, 'num_bytes_recovered': 5881856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2949}, {'shards': '2', 'objects': 3663}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6586", 'reported_seq': 6704, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:22.399342+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:22.399342+0000', 'last_peered': '2024-09-05T13:14:22.399342+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:22.399342+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 6586, 'log_dups_size': 0, 'ondisk_log_size': 6586, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26951680, 'num_objects': 6580, 'num_object_clones': 0, 'num_object_copies': 13160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4095, 'num_write_kb': 16380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 2193}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:26.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6633", 'reported_seq': 6768, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:21.479081+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:21.479081+0000', 'last_peered': '2024-09-05T13:14:21.479081+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:21.479081+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 6633, 'log_dups_size': 0, 'ondisk_log_size': 6633, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27160576, 'num_objects': 6631, 'num_object_clones': 0, 'num_object_copies': 13262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2784, 'num_write_kb': 11136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2212}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:26.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:26.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6703", 'reported_seq': 6885, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:21.583746+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:21.583746+0000', 'last_peered': '2024-09-05T13:14:21.583746+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:21.583746+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 6703, 'log_dups_size': 0, 'ondisk_log_size': 6703, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27422720, 'num_objects': 6695, 'num_object_clones': 0, 'num_object_copies': 13390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2602, 'num_write_kb': 10408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2251}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:26.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:26.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6734", 'reported_seq': 6868, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:21.614066+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:21.614066+0000', 'last_peered': '2024-09-05T13:14:21.614066+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:21.614066+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 6734, 'log_dups_size': 0, 'ondisk_log_size': 6734, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27566080, 'num_objects': 6730, 'num_object_clones': 0, 'num_object_copies': 13460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2864, 'num_write_kb': 11456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2164}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:26.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:26.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6797", 'reported_seq': 6932, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:21.605003+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:21.605003+0000', 'last_peered': '2024-09-05T13:14:21.605003+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:21.605003+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 6797, 'log_dups_size': 0, 'ondisk_log_size': 6797, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27791843, 'num_objects': 6786, 'num_object_clones': 0, 'num_object_copies': 13572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6786, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3465, 'num_write_kb': 13860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2221}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:26.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:27.047 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/213734 objects degraded (22.976%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":193,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":106867,"data_bytes":437715446,"bytes_used":1474899968,"bytes_avail":288435392512,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":213734,"degraded_ratio":0.22976222781588329,"misplaced_objects":2949,"misplaced_total":213734,"misplaced_ratio":0.01379752402519019,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":77107,"recovering_keys_per_sec":0,"num_objects_recovered":226,"num_bytes_recovered":925696,"num_keys_recovered":0,"write_bytes_sec":1786434,"read_op_per_sec":0,"write_op_per_sec":436},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:14:27.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:27.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:27.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 114 20 109607 109587 3.75392 1.84375 0.0455571 0.0208037 2024-09-05T13:14:27.636 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:27.637 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:27.661 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-05T13:14:27.544625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448385526,"num_objects":109472,"num_object_clones":0,"num_object_copies":218944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":109472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65271,"num_write_kb":261078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1569,"num_bytes_recovered":6422547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109554,"ondisk_log_size":109554,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480892,"kb_used_data":876364,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":281634628,"statfs":{"total":289910292480,"available":288393859072,"internally_reserved":0,"allocated":897396736,"data_stored":896573826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":618974214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,6,11,9,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32116736,"num_objects":7841,"num_object_clones":0,"num_object_copies":15682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-355,"num_objects_unfound":0,"num_objects_dirty":7841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7841,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005100"},"pg_stats":[{"pgid":"2.f","version":"43'6790","reported_seq":6860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.489941+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:26.489941+0000","last_peered":"2024-09-05T13:14:26.489941+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:26.489941+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6790,"log_dups_size":0,"ondisk_log_size":6790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6790,"num_write_kb":27160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2369},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6814","reported_seq":6913,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:27.391143+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:27.391143+0000","last_peered":"2024-09-05T13:14:27.391143+0000","last_clean":"2024-09-05T13:14:27.391143+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:27.391143+0000","last_undegraded":"2024-09-05T13:14:27.391143+0000","last_fullsized":"2024-09-05T13:14:27.391143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27885568,"num_objects":6808,"num_object_clones":0,"num_object_copies":13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6904","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.426063+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:27.426063+0000","last_peered":"2024-09-05T13:14:27.426063+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:27.426063+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6904,"log_dups_size":0,"ondisk_log_size":6904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28262400,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4715,"num_write_kb":18860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6915","reported_seq":6979,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:27.389045+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:27.389045+0000","last_peered":"2024-09-05T13:14:27.389045+0000","last_clean":"2024-09-05T13:14:27.389045+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:27.389045+0000","last_undegraded":"2024-09-05T13:14:27.389045+0000","last_fullsized":"2024-09-05T13:14:27.389045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6915,"log_dups_size":0,"ondisk_log_size":6915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28323840,"num_objects":6915,"num_object_clones":0,"num_object_copies":13830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6865","reported_seq":6994,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:26.599370+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:26.599370+0000","last_peered":"2024-09-05T13:14:26.599370+0000","last_clean":"2024-09-05T13:14:26.599370+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:26.599370+0000","last_undegraded":"2024-09-05T13:14:26.599370+0000","last_fullsized":"2024-09-05T13:14:26.599370+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6865,"log_dups_size":0,"ondisk_log_size":6865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090368,"num_objects":6858,"num_object_clones":0,"num_object_copies":13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3405,"num_write_kb":13620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6862","reported_seq":6995,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.633296+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:26.633296+0000","last_peered":"2024-09-05T13:14:26.633296+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:26.633296+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6862,"log_dups_size":0,"ondisk_log_size":6862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28086272,"num_objects":6857,"num_object_clones":0,"num_object_copies":13714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3317,"num_write_kb":13268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2406},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6793","reported_seq":6928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.571069+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:26.571069+0000","last_peered":"2024-09-05T13:14:26.571069+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:26.571069+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27799552,"num_objects":6787,"num_object_clones":0,"num_object_copies":13574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2351},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6892","reported_seq":7019,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:26.519927+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:26.519927+0000","last_peered":"2024-09-05T13:14:26.519927+0000","last_clean":"2024-09-05T13:14:26.519927+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:26.519927+0000","last_undegraded":"2024-09-05T13:14:26.519927+0000","last_fullsized":"2024-09-05T13:14:26.519927+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6892,"log_dups_size":0,"ondisk_log_size":6892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28221440,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3502,"num_write_kb":14008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6835","reported_seq":6964,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.434314+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:27.434314+0000","last_peered":"2024-09-05T13:14:27.434314+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:27.434314+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6835,"log_dups_size":0,"ondisk_log_size":6835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4489,"num_write_kb":17956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6799","reported_seq":6932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.525421+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:26.525421+0000","last_peered":"2024-09-05T13:14:26.525421+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:26.525421+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6799,"log_dups_size":0,"ondisk_log_size":6799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3166,"num_write_kb":12664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2337},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6794","reported_seq":10036,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.435071+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:27.435071+0000","last_peered":"2024-09-05T13:14:27.435071+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:27.435071+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6794,"log_dups_size":0,"ondisk_log_size":6794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27807744,"num_objects":6789,"num_object_clones":0,"num_object_copies":13578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":6789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4389,"num_write_kb":17556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1565,"num_bytes_recovered":6410240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2820},{"shards":"2","objects":3967}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6748","reported_seq":6866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.419843+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:27.419843+0000","last_peered":"2024-09-05T13:14:27.419843+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:27.419843+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615232,"num_objects":6742,"num_object_clones":0,"num_object_copies":13484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4257,"num_write_kb":17028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6806","reported_seq":6941,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.591138+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:26.591138+0000","last_peered":"2024-09-05T13:14:26.591138+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:26.591138+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2957,"num_write_kb":11828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2385},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6877","reported_seq":7059,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.585927+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:26.585927+0000","last_peered":"2024-09-05T13:14:26.585927+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:26.585927+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6877,"log_dups_size":0,"ondisk_log_size":6877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":11104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2425},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6887","reported_seq":7017,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.628902+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:26.628902+0000","last_peered":"2024-09-05T13:14:26.628902+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:26.628902+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28192768,"num_objects":6883,"num_object_clones":0,"num_object_copies":13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3017,"num_write_kb":12068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6970","reported_seq":7105,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.619224+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:26.619224+0000","last_peered":"2024-09-05T13:14:26.619224+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:26.619224+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6970,"log_dups_size":0,"ondisk_log_size":6970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28500451,"num_objects":6959,"num_object_clones":0,"num_object_copies":13918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3638,"num_write_kb":14552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2394},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448385507,"num_objects":109470,"num_object_clones":0,"num_object_copies":218940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":109470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65269,"num_write_kb":261076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1568,"num_bytes_recovered":6422528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":898912256,"data_stored":898821931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109551,"ondisk_log_size":109551,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570228,"kb_used_data":362212,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93801612,"statfs":{"total":96636764160,"available":96052850688,"internally_reserved":0,"allocated":370905088,"data_stored":370659998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":212987565},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669104,"kb_used_data":398688,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93702736,"statfs":{"total":96636764160,"available":95951601664,"internally_reserved":0,"allocated":408256512,"data_stored":408007345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241560,"kb_used_data":115464,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94130280,"statfs":{"total":96636764160,"available":96389406720,"internally_reserved":0,"allocated":118235136,"data_stored":117906483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129101487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409116672,"data_stored":409113059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":118964224,"data_stored":118881125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370831360,"data_stored":370827747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:28.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 115 20 110117 110097 3.7386 1.99219 0.0238478 0.0208865 2024-09-05T13:14:29.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 116 20 110657 110637 3.72456 2.10938 0.0234922 0.0209687 2024-09-05T13:14:30.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 117 20 111200 111180 3.71085 2.12109 0.019848 0.0210448 2024-09-05T13:14:30.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:31.208 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:31.211 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:31.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 118 20 111710 111690 3.69629 1.99219 0.0504675 0.0211272 2024-09-05T13:14:31.238 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-05T13:14:29.545223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448385526,"num_objects":109472,"num_object_clones":0,"num_object_copies":218944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":109472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65271,"num_write_kb":261078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1569,"num_bytes_recovered":6422547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109554,"ondisk_log_size":109554,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493360,"kb_used_data":884672,"kb_used_omap":12,"kb_used_meta":608627,"kb_avail":281622160,"statfs":{"total":289910292480,"available":288381091840,"internally_reserved":0,"allocated":905904128,"data_stored":905081218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":623234054},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,12,6,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21323776,"num_objects":5206,"num_object_clones":0,"num_object_copies":10412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":5206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005047"},"pg_stats":[{"pgid":"2.f","version":"43'6790","reported_seq":6860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.489941+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:26.489941+0000","last_peered":"2024-09-05T13:14:26.489941+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:26.489941+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6790,"log_dups_size":0,"ondisk_log_size":6790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6790,"num_write_kb":27160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2369},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6814","reported_seq":6913,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:27.391143+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:27.391143+0000","last_peered":"2024-09-05T13:14:27.391143+0000","last_clean":"2024-09-05T13:14:27.391143+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:27.391143+0000","last_undegraded":"2024-09-05T13:14:27.391143+0000","last_fullsized":"2024-09-05T13:14:27.391143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27885568,"num_objects":6808,"num_object_clones":0,"num_object_copies":13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6904","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.426063+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:27.426063+0000","last_peered":"2024-09-05T13:14:27.426063+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:27.426063+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6904,"log_dups_size":0,"ondisk_log_size":6904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28262400,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4715,"num_write_kb":18860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6915","reported_seq":6979,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:27.389045+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:27.389045+0000","last_peered":"2024-09-05T13:14:27.389045+0000","last_clean":"2024-09-05T13:14:27.389045+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:27.389045+0000","last_undegraded":"2024-09-05T13:14:27.389045+0000","last_fullsized":"2024-09-05T13:14:27.389045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6915,"log_dups_size":0,"ondisk_log_size":6915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28323840,"num_objects":6915,"num_object_clones":0,"num_object_copies":13830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6865","reported_seq":6994,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:26.599370+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:26.599370+0000","last_peered":"2024-09-05T13:14:26.599370+0000","last_clean":"2024-09-05T13:14:26.599370+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:26.599370+0000","last_undegraded":"2024-09-05T13:14:26.599370+0000","last_fullsized":"2024-09-05T13:14:26.599370+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6865,"log_dups_size":0,"ondisk_log_size":6865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090368,"num_objects":6858,"num_object_clones":0,"num_object_copies":13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3405,"num_write_kb":13620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6862","reported_seq":6995,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.633296+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:26.633296+0000","last_peered":"2024-09-05T13:14:26.633296+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:26.633296+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6862,"log_dups_size":0,"ondisk_log_size":6862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28086272,"num_objects":6857,"num_object_clones":0,"num_object_copies":13714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3317,"num_write_kb":13268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2406},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6793","reported_seq":6928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.571069+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:26.571069+0000","last_peered":"2024-09-05T13:14:26.571069+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:26.571069+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27799552,"num_objects":6787,"num_object_clones":0,"num_object_copies":13574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2351},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6892","reported_seq":7019,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:26.519927+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:26.519927+0000","last_peered":"2024-09-05T13:14:26.519927+0000","last_clean":"2024-09-05T13:14:26.519927+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:26.519927+0000","last_undegraded":"2024-09-05T13:14:26.519927+0000","last_fullsized":"2024-09-05T13:14:26.519927+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6892,"log_dups_size":0,"ondisk_log_size":6892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28221440,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3502,"num_write_kb":14008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6835","reported_seq":6964,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.434314+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:27.434314+0000","last_peered":"2024-09-05T13:14:27.434314+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:27.434314+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6835,"log_dups_size":0,"ondisk_log_size":6835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4489,"num_write_kb":17956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6799","reported_seq":6932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.525421+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:26.525421+0000","last_peered":"2024-09-05T13:14:26.525421+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:26.525421+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6799,"log_dups_size":0,"ondisk_log_size":6799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3166,"num_write_kb":12664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2337},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6794","reported_seq":10036,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.435071+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:27.435071+0000","last_peered":"2024-09-05T13:14:27.435071+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:27.435071+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6794,"log_dups_size":0,"ondisk_log_size":6794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27807744,"num_objects":6789,"num_object_clones":0,"num_object_copies":13578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":6789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4389,"num_write_kb":17556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1565,"num_bytes_recovered":6410240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2820},{"shards":"2","objects":3967}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6748","reported_seq":6866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.419843+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:27.419843+0000","last_peered":"2024-09-05T13:14:27.419843+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:27.419843+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615232,"num_objects":6742,"num_object_clones":0,"num_object_copies":13484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4257,"num_write_kb":17028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6806","reported_seq":6941,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.591138+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:26.591138+0000","last_peered":"2024-09-05T13:14:26.591138+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:26.591138+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2957,"num_write_kb":11828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2385},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6877","reported_seq":7059,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.585927+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:26.585927+0000","last_peered":"2024-09-05T13:14:26.585927+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:26.585927+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6877,"log_dups_size":0,"ondisk_log_size":6877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":11104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2425},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6887","reported_seq":7017,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.628902+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:26.628902+0000","last_peered":"2024-09-05T13:14:26.628902+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:26.628902+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28192768,"num_objects":6883,"num_object_clones":0,"num_object_copies":13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3017,"num_write_kb":12068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6970","reported_seq":7105,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.619224+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:26.619224+0000","last_peered":"2024-09-05T13:14:26.619224+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:26.619224+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6970,"log_dups_size":0,"ondisk_log_size":6970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28500451,"num_objects":6959,"num_object_clones":0,"num_object_copies":13918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3638,"num_write_kb":14552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2394},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448385507,"num_objects":109470,"num_object_clones":0,"num_object_copies":218940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":109470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65269,"num_write_kb":261076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1568,"num_bytes_recovered":6422528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907456512,"data_stored":907366187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109551,"ondisk_log_size":109551,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570228,"kb_used_data":362212,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93801612,"statfs":{"total":96636764160,"available":96052850688,"internally_reserved":0,"allocated":370905088,"data_stored":370659998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":212987565},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669104,"kb_used_data":398688,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93702736,"statfs":{"total":96636764160,"available":95951601664,"internally_reserved":0,"allocated":408256512,"data_stored":408007345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254028,"kb_used_data":123772,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94117812,"statfs":{"total":96636764160,"available":96376639488,"internally_reserved":0,"allocated":126742528,"data_stored":126413875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133361327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409116672,"data_stored":409113059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127508480,"data_stored":127425381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370831360,"data_stored":370827747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:31.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:31.800 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:31.824 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-05T13:14:31.549062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448385526,"num_objects":109472,"num_object_clones":0,"num_object_copies":218944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":109472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65271,"num_write_kb":261078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1569,"num_bytes_recovered":6422547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109554,"ondisk_log_size":109554,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493360,"kb_used_data":884672,"kb_used_omap":12,"kb_used_meta":608627,"kb_avail":281622160,"statfs":{"total":289910292480,"available":288381091840,"internally_reserved":0,"allocated":905904128,"data_stored":905081218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":623234054},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,12,6,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21323776,"num_objects":5206,"num_object_clones":0,"num_object_copies":10412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":5206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008022"},"pg_stats":[{"pgid":"2.f","version":"43'6790","reported_seq":6860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.489941+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:26.489941+0000","last_peered":"2024-09-05T13:14:26.489941+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:26.489941+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6790,"log_dups_size":0,"ondisk_log_size":6790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6790,"num_write_kb":27160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2369},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6814","reported_seq":6913,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:27.391143+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:27.391143+0000","last_peered":"2024-09-05T13:14:27.391143+0000","last_clean":"2024-09-05T13:14:27.391143+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:27.391143+0000","last_undegraded":"2024-09-05T13:14:27.391143+0000","last_fullsized":"2024-09-05T13:14:27.391143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27885568,"num_objects":6808,"num_object_clones":0,"num_object_copies":13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6904","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.426063+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:27.426063+0000","last_peered":"2024-09-05T13:14:27.426063+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:27.426063+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6904,"log_dups_size":0,"ondisk_log_size":6904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28262400,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4715,"num_write_kb":18860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6915","reported_seq":6979,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:27.389045+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:27.389045+0000","last_peered":"2024-09-05T13:14:27.389045+0000","last_clean":"2024-09-05T13:14:27.389045+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:27.389045+0000","last_undegraded":"2024-09-05T13:14:27.389045+0000","last_fullsized":"2024-09-05T13:14:27.389045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6915,"log_dups_size":0,"ondisk_log_size":6915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28323840,"num_objects":6915,"num_object_clones":0,"num_object_copies":13830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6865","reported_seq":6994,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:26.599370+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:26.599370+0000","last_peered":"2024-09-05T13:14:26.599370+0000","last_clean":"2024-09-05T13:14:26.599370+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:26.599370+0000","last_undegraded":"2024-09-05T13:14:26.599370+0000","last_fullsized":"2024-09-05T13:14:26.599370+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6865,"log_dups_size":0,"ondisk_log_size":6865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090368,"num_objects":6858,"num_object_clones":0,"num_object_copies":13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3405,"num_write_kb":13620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6862","reported_seq":6995,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.633296+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:26.633296+0000","last_peered":"2024-09-05T13:14:26.633296+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:26.633296+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6862,"log_dups_size":0,"ondisk_log_size":6862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28086272,"num_objects":6857,"num_object_clones":0,"num_object_copies":13714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3317,"num_write_kb":13268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2406},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6793","reported_seq":6928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.571069+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:26.571069+0000","last_peered":"2024-09-05T13:14:26.571069+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:26.571069+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27799552,"num_objects":6787,"num_object_clones":0,"num_object_copies":13574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2351},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6892","reported_seq":7019,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:26.519927+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:26.519927+0000","last_peered":"2024-09-05T13:14:26.519927+0000","last_clean":"2024-09-05T13:14:26.519927+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:26.519927+0000","last_undegraded":"2024-09-05T13:14:26.519927+0000","last_fullsized":"2024-09-05T13:14:26.519927+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6892,"log_dups_size":0,"ondisk_log_size":6892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28221440,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3502,"num_write_kb":14008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6835","reported_seq":6964,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.434314+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:27.434314+0000","last_peered":"2024-09-05T13:14:27.434314+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:27.434314+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6835,"log_dups_size":0,"ondisk_log_size":6835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4489,"num_write_kb":17956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6799","reported_seq":6932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.525421+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:26.525421+0000","last_peered":"2024-09-05T13:14:26.525421+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:26.525421+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6799,"log_dups_size":0,"ondisk_log_size":6799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3166,"num_write_kb":12664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2337},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6794","reported_seq":10036,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.435071+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:27.435071+0000","last_peered":"2024-09-05T13:14:27.435071+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:27.435071+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6794,"log_dups_size":0,"ondisk_log_size":6794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27807744,"num_objects":6789,"num_object_clones":0,"num_object_copies":13578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":6789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4389,"num_write_kb":17556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1565,"num_bytes_recovered":6410240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2820},{"shards":"2","objects":3967}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6748","reported_seq":6866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.419843+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:27.419843+0000","last_peered":"2024-09-05T13:14:27.419843+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:27.419843+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615232,"num_objects":6742,"num_object_clones":0,"num_object_copies":13484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4257,"num_write_kb":17028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6806","reported_seq":6941,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.591138+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:26.591138+0000","last_peered":"2024-09-05T13:14:26.591138+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:26.591138+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2957,"num_write_kb":11828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2385},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6877","reported_seq":7059,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.585927+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:26.585927+0000","last_peered":"2024-09-05T13:14:26.585927+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:26.585927+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6877,"log_dups_size":0,"ondisk_log_size":6877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":11104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2425},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6887","reported_seq":7017,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.628902+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:26.628902+0000","last_peered":"2024-09-05T13:14:26.628902+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:26.628902+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28192768,"num_objects":6883,"num_object_clones":0,"num_object_copies":13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3017,"num_write_kb":12068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6970","reported_seq":7105,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.619224+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:26.619224+0000","last_peered":"2024-09-05T13:14:26.619224+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:26.619224+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6970,"log_dups_size":0,"ondisk_log_size":6970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28500451,"num_objects":6959,"num_object_clones":0,"num_object_copies":13918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3638,"num_write_kb":14552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2394},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448385507,"num_objects":109470,"num_object_clones":0,"num_object_copies":218940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":109470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65269,"num_write_kb":261076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1568,"num_bytes_recovered":6422528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907456512,"data_stored":907366187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109551,"ondisk_log_size":109551,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570228,"kb_used_data":362212,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93801612,"statfs":{"total":96636764160,"available":96052850688,"internally_reserved":0,"allocated":370905088,"data_stored":370659998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":212987565},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669104,"kb_used_data":398688,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93702736,"statfs":{"total":96636764160,"available":95951601664,"internally_reserved":0,"allocated":408256512,"data_stored":408007345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254028,"kb_used_data":123772,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94117812,"statfs":{"total":96636764160,"available":96376639488,"internally_reserved":0,"allocated":126742528,"data_stored":126413875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133361327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409116672,"data_stored":409113059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127508480,"data_stored":127425381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370831360,"data_stored":370827747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6790", 'reported_seq': 6860, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:26.489941+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:26.489941+0000', 'last_peered': '2024-09-05T13:14:26.489941+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:26.489941+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 6790, 'log_dups_size': 0, 'ondisk_log_size': 6790, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27811840, 'num_objects': 6790, 'num_object_clones': 0, 'num_object_copies': 13580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6790, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6790, 'num_write_kb': 27160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2369}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6904", 'reported_seq': 7026, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:27.426063+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:27.426063+0000', 'last_peered': '2024-09-05T13:14:27.426063+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:27.426063+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 6904, 'log_dups_size': 0, 'ondisk_log_size': 6904, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28262400, 'num_objects': 6900, 'num_object_clones': 0, 'num_object_copies': 13800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4715, 'num_write_kb': 18860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 2408}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6862", 'reported_seq': 6995, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:26.633296+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:26.633296+0000', 'last_peered': '2024-09-05T13:14:26.633296+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:26.633296+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 6862, 'log_dups_size': 0, 'ondisk_log_size': 6862, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28086272, 'num_objects': 6857, 'num_object_clones': 0, 'num_object_copies': 13714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6857, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3317, 'num_write_kb': 13268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2406}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6793", 'reported_seq': 6928, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:26.571069+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:26.571069+0000', 'last_peered': '2024-09-05T13:14:26.571069+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:26.571069+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 6793, 'log_dups_size': 0, 'ondisk_log_size': 6793, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27799552, 'num_objects': 6787, 'num_object_clones': 0, 'num_object_copies': 13574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6787, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3112, 'num_write_kb': 12448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2351}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:31.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6835", 'reported_seq': 6964, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:27.434314+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:27.434314+0000', 'last_peered': '2024-09-05T13:14:27.434314+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:27.434314+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 6835, 'log_dups_size': 0, 'ondisk_log_size': 6835, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27967488, 'num_objects': 6828, 'num_object_clones': 0, 'num_object_copies': 13656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4489, 'num_write_kb': 17956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 2377}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6799", 'reported_seq': 6932, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:26.525421+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:26.525421+0000', 'last_peered': '2024-09-05T13:14:26.525421+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:26.525421+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 6799, 'log_dups_size': 0, 'ondisk_log_size': 6799, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27815936, 'num_objects': 6791, 'num_object_clones': 0, 'num_object_copies': 13582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3166, 'num_write_kb': 12664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2337}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6794", 'reported_seq': 10036, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:27.435071+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:27.435071+0000', 'last_peered': '2024-09-05T13:14:27.435071+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:27.435071+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 6794, 'log_dups_size': 0, 'ondisk_log_size': 6794, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27807744, 'num_objects': 6789, 'num_object_clones': 0, 'num_object_copies': 13578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2820, 'num_objects_unfound': 0, 'num_objects_dirty': 6789, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4389, 'num_write_kb': 17556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1565, 'num_bytes_recovered': 6410240, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2820}, {'shards': '2', 'objects': 3967}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6748", 'reported_seq': 6866, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:27.419843+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:27.419843+0000', 'last_peered': '2024-09-05T13:14:27.419843+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:27.419843+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 6748, 'log_dups_size': 0, 'ondisk_log_size': 6748, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27615232, 'num_objects': 6742, 'num_object_clones': 0, 'num_object_copies': 13484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4257, 'num_write_kb': 17028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 2355}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6806", 'reported_seq': 6941, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:26.591138+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:26.591138+0000', 'last_peered': '2024-09-05T13:14:26.591138+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:26.591138+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 6806, 'log_dups_size': 0, 'ondisk_log_size': 6806, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27869184, 'num_objects': 6804, 'num_object_clones': 0, 'num_object_copies': 13608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2957, 'num_write_kb': 11828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2385}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:31.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6877", 'reported_seq': 7059, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:26.585927+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:26.585927+0000', 'last_peered': '2024-09-05T13:14:26.585927+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:26.585927+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 6877, 'log_dups_size': 0, 'ondisk_log_size': 6877, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28135424, 'num_objects': 6869, 'num_object_clones': 0, 'num_object_copies': 13738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2776, 'num_write_kb': 11104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2425}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:31.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:31.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6887", 'reported_seq': 7017, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:26.628902+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:26.628902+0000', 'last_peered': '2024-09-05T13:14:26.628902+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:26.628902+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 6887, 'log_dups_size': 0, 'ondisk_log_size': 6887, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28192768, 'num_objects': 6883, 'num_object_clones': 0, 'num_object_copies': 13766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6883, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3017, 'num_write_kb': 12068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2317}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:31.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:31.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6970", 'reported_seq': 7105, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:26.619224+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:26.619224+0000', 'last_peered': '2024-09-05T13:14:26.619224+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:26.619224+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 6970, 'log_dups_size': 0, 'ondisk_log_size': 6970, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28500451, 'num_objects': 6959, 'num_object_clones': 0, 'num_object_copies': 13918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6959, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3638, 'num_write_kb': 14552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2394}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:31.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:32.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 119 20 112181 112161 3.68069 1.83984 0.0280008 0.0212184 2024-09-05T13:14:32.551 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:32.551 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/218944 objects degraded (22.429%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":199,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109472,"data_bytes":448385526,"bytes_used":1529200640,"bytes_avail":288381091840,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":218944,"degraded_ratio":0.22429479684302836,"misplaced_objects":2820,"misplaced_total":218944,"misplaced_ratio":0.01288000584624379,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83911,"recovering_keys_per_sec":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"write_bytes_sec":1775794,"read_op_per_sec":0,"write_op_per_sec":433},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:14:32.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:32.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:33.145 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:33.145 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:33.172 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-05T13:14:31.549062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448385526,"num_objects":109472,"num_object_clones":0,"num_object_copies":218944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":109472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65271,"num_write_kb":261078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1569,"num_bytes_recovered":6422547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109554,"ondisk_log_size":109554,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493360,"kb_used_data":884672,"kb_used_omap":12,"kb_used_meta":608627,"kb_avail":281622160,"statfs":{"total":289910292480,"available":288381091840,"internally_reserved":0,"allocated":905904128,"data_stored":905081218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":623234054},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,12,6,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21323776,"num_objects":5206,"num_object_clones":0,"num_object_copies":10412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":5206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008022"},"pg_stats":[{"pgid":"2.f","version":"43'6790","reported_seq":6860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.489941+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:26.489941+0000","last_peered":"2024-09-05T13:14:26.489941+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:26.489941+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6790,"log_dups_size":0,"ondisk_log_size":6790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6790,"num_write_kb":27160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2369},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6814","reported_seq":6913,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:27.391143+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:27.391143+0000","last_peered":"2024-09-05T13:14:27.391143+0000","last_clean":"2024-09-05T13:14:27.391143+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:27.391143+0000","last_undegraded":"2024-09-05T13:14:27.391143+0000","last_fullsized":"2024-09-05T13:14:27.391143+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27885568,"num_objects":6808,"num_object_clones":0,"num_object_copies":13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6904","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.426063+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:27.426063+0000","last_peered":"2024-09-05T13:14:27.426063+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:27.426063+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":6904,"log_dups_size":0,"ondisk_log_size":6904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28262400,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4715,"num_write_kb":18860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6915","reported_seq":6979,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:27.389045+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:27.389045+0000","last_peered":"2024-09-05T13:14:27.389045+0000","last_clean":"2024-09-05T13:14:27.389045+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:27.389045+0000","last_undegraded":"2024-09-05T13:14:27.389045+0000","last_fullsized":"2024-09-05T13:14:27.389045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6915,"log_dups_size":0,"ondisk_log_size":6915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28323840,"num_objects":6915,"num_object_clones":0,"num_object_copies":13830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6915,"num_write_kb":27660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6865","reported_seq":6994,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:26.599370+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:26.599370+0000","last_peered":"2024-09-05T13:14:26.599370+0000","last_clean":"2024-09-05T13:14:26.599370+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:26.599370+0000","last_undegraded":"2024-09-05T13:14:26.599370+0000","last_fullsized":"2024-09-05T13:14:26.599370+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":6865,"log_dups_size":0,"ondisk_log_size":6865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090368,"num_objects":6858,"num_object_clones":0,"num_object_copies":13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3405,"num_write_kb":13620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6862","reported_seq":6995,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.633296+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:26.633296+0000","last_peered":"2024-09-05T13:14:26.633296+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:26.633296+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":6862,"log_dups_size":0,"ondisk_log_size":6862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28086272,"num_objects":6857,"num_object_clones":0,"num_object_copies":13714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3317,"num_write_kb":13268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2406},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6793","reported_seq":6928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.571069+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:26.571069+0000","last_peered":"2024-09-05T13:14:26.571069+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:26.571069+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27799552,"num_objects":6787,"num_object_clones":0,"num_object_copies":13574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":12448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2351},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6892","reported_seq":7019,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:26.519927+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:26.519927+0000","last_peered":"2024-09-05T13:14:26.519927+0000","last_clean":"2024-09-05T13:14:26.519927+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:26.519927+0000","last_undegraded":"2024-09-05T13:14:26.519927+0000","last_fullsized":"2024-09-05T13:14:26.519927+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":6892,"log_dups_size":0,"ondisk_log_size":6892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28221440,"num_objects":6890,"num_object_clones":0,"num_object_copies":13780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3502,"num_write_kb":14008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6835","reported_seq":6964,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.434314+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:27.434314+0000","last_peered":"2024-09-05T13:14:27.434314+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:27.434314+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6835,"log_dups_size":0,"ondisk_log_size":6835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4489,"num_write_kb":17956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6799","reported_seq":6932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.525421+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:26.525421+0000","last_peered":"2024-09-05T13:14:26.525421+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:26.525421+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6799,"log_dups_size":0,"ondisk_log_size":6799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3166,"num_write_kb":12664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2337},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6794","reported_seq":10036,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.435071+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:27.435071+0000","last_peered":"2024-09-05T13:14:27.435071+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:27.435071+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6794,"log_dups_size":0,"ondisk_log_size":6794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27807744,"num_objects":6789,"num_object_clones":0,"num_object_copies":13578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":6789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4389,"num_write_kb":17556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1565,"num_bytes_recovered":6410240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2820},{"shards":"2","objects":3967}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6748","reported_seq":6866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:27.419843+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:27.419843+0000","last_peered":"2024-09-05T13:14:27.419843+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:27.419843+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6748,"log_dups_size":0,"ondisk_log_size":6748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27615232,"num_objects":6742,"num_object_clones":0,"num_object_copies":13484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4257,"num_write_kb":17028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6806","reported_seq":6941,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.591138+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:26.591138+0000","last_peered":"2024-09-05T13:14:26.591138+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:26.591138+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2957,"num_write_kb":11828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2385},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6877","reported_seq":7059,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.585927+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:26.585927+0000","last_peered":"2024-09-05T13:14:26.585927+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:26.585927+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":6877,"log_dups_size":0,"ondisk_log_size":6877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":11104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2425},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6887","reported_seq":7017,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.628902+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:26.628902+0000","last_peered":"2024-09-05T13:14:26.628902+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:26.628902+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":6887,"log_dups_size":0,"ondisk_log_size":6887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28192768,"num_objects":6883,"num_object_clones":0,"num_object_copies":13766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3017,"num_write_kb":12068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6970","reported_seq":7105,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:26.619224+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:26.619224+0000","last_peered":"2024-09-05T13:14:26.619224+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:26.619224+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":6970,"log_dups_size":0,"ondisk_log_size":6970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28500451,"num_objects":6959,"num_object_clones":0,"num_object_copies":13918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3638,"num_write_kb":14552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2394},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448385507,"num_objects":109470,"num_object_clones":0,"num_object_copies":218940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2820,"num_objects_unfound":0,"num_objects_dirty":109470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65269,"num_write_kb":261076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1568,"num_bytes_recovered":6422528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907456512,"data_stored":907366187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109551,"ondisk_log_size":109551,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570228,"kb_used_data":362212,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93801612,"statfs":{"total":96636764160,"available":96052850688,"internally_reserved":0,"allocated":370905088,"data_stored":370659998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":212987565},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669104,"kb_used_data":398688,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93702736,"statfs":{"total":96636764160,"available":95951601664,"internally_reserved":0,"allocated":408256512,"data_stored":408007345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254028,"kb_used_data":123772,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94117812,"statfs":{"total":96636764160,"available":96376639488,"internally_reserved":0,"allocated":126742528,"data_stored":126413875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133361327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409116672,"data_stored":409113059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127508480,"data_stored":127425381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370831360,"data_stored":370827747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:33.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:14:33.219476+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0213062 2024-09-05T13:14:33.220 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:14:33.220 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 120 20 112661 112641 3.66565 1.875 0.0863036 0.0213062 2024-09-05T13:14:34.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 121 20 113202 113182 3.65282 2.11328 0.0129303 0.0213809 2024-09-05T13:14:35.220 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 122 20 113733 113713 3.63988 2.07422 0.0296676 0.0214556 2024-09-05T13:14:36.174 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:36.220 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 123 20 114266 114246 3.62722 2.08203 0.0278495 0.0215301 2024-09-05T13:14:36.718 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:36.743 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-05T13:14:35.550665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":458985974,"num_objects":112060,"num_object_clones":0,"num_object_copies":224120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2700,"num_objects_unfound":0,"num_objects_dirty":112060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67859,"num_write_kb":271430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1689,"num_bytes_recovered":6914067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112142,"ondisk_log_size":112142,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532044,"kb_used_data":905308,"kb_used_omap":12,"kb_used_meta":626675,"kb_avail":281583476,"statfs":{"total":289910292480,"available":288341479424,"internally_reserved":0,"allocated":927035392,"data_stored":926212482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":641715206},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,6,2,10,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21270528,"num_objects":5193,"num_object_clones":0,"num_object_copies":10386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":5193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5193,"num_write_kb":20772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007275"},"pg_stats":[{"pgid":"2.f","version":"43'6953","reported_seq":7021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.613515+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:31.613515+0000","last_peered":"2024-09-05T13:14:31.613515+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:31.613515+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6953,"log_dups_size":0,"ondisk_log_size":6953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28479488,"num_objects":6953,"num_object_clones":0,"num_object_copies":13906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6953,"num_write_kb":27812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2532},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6990","reported_seq":7090,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:32.444969+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:32.444969+0000","last_peered":"2024-09-05T13:14:32.444969+0000","last_clean":"2024-09-05T13:14:32.444969+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:32.444969+0000","last_undegraded":"2024-09-05T13:14:32.444969+0000","last_fullsized":"2024-09-05T13:14:32.444969+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6990,"log_dups_size":0,"ondisk_log_size":6990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28606464,"num_objects":6984,"num_object_clones":0,"num_object_copies":13968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5000,"num_write_kb":20000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7065","reported_seq":7186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:32.436377+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:32.436377+0000","last_peered":"2024-09-05T13:14:32.436377+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:32.436377+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7065,"log_dups_size":0,"ondisk_log_size":7065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28921856,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4876,"num_write_kb":19504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2569}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7066","reported_seq":7131,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:32.469320+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:32.469320+0000","last_peered":"2024-09-05T13:14:32.469320+0000","last_clean":"2024-09-05T13:14:32.469320+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:32.469320+0000","last_undegraded":"2024-09-05T13:14:32.469320+0000","last_fullsized":"2024-09-05T13:14:32.469320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7066,"log_dups_size":0,"ondisk_log_size":7066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28942336,"num_objects":7066,"num_object_clones":0,"num_object_copies":14132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7066,"num_write_kb":28264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7023","reported_seq":7152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:31.630303+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:31.630303+0000","last_peered":"2024-09-05T13:14:31.630303+0000","last_clean":"2024-09-05T13:14:31.630303+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:31.630303+0000","last_undegraded":"2024-09-05T13:14:31.630303+0000","last_fullsized":"2024-09-05T13:14:31.630303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7023,"log_dups_size":0,"ondisk_log_size":7023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28737536,"num_objects":7016,"num_object_clones":0,"num_object_copies":14032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3563,"num_write_kb":14252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7042","reported_seq":7175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.614524+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:31.614524+0000","last_peered":"2024-09-05T13:14:31.614524+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:31.614524+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7042,"log_dups_size":0,"ondisk_log_size":7042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3497,"num_write_kb":13988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2586},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6942","reported_seq":7076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.519133+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:31.519133+0000","last_peered":"2024-09-05T13:14:31.519133+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:31.519133+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6942,"log_dups_size":0,"ondisk_log_size":6942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28409856,"num_objects":6936,"num_object_clones":0,"num_object_copies":13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3261,"num_write_kb":13044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2500},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7044","reported_seq":7171,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:31.639913+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:31.639913+0000","last_peered":"2024-09-05T13:14:31.639913+0000","last_clean":"2024-09-05T13:14:31.639913+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:31.639913+0000","last_undegraded":"2024-09-05T13:14:31.639913+0000","last_fullsized":"2024-09-05T13:14:31.639913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7044,"log_dups_size":0,"ondisk_log_size":7044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28844032,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3654,"num_write_kb":14616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6998","reported_seq":7128,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:32.464380+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:32.464380+0000","last_peered":"2024-09-05T13:14:32.464380+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:32.464380+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6998,"log_dups_size":0,"ondisk_log_size":6998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28635136,"num_objects":6991,"num_object_clones":0,"num_object_copies":13982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":18608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2540}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6948","reported_seq":7081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.566156+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:31.566156+0000","last_peered":"2024-09-05T13:14:31.566156+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:31.566156+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6948,"log_dups_size":0,"ondisk_log_size":6948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3315,"num_write_kb":13260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2486},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6969","reported_seq":10451,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:32.439001+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:32.439001+0000","last_peered":"2024-09-05T13:14:32.439001+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:32.439001+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6969,"log_dups_size":0,"ondisk_log_size":6969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2700,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4564,"num_write_kb":18256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1685,"num_bytes_recovered":6901760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2700},{"shards":"2","objects":4262}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6909","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:32.410016+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:32.410016+0000","last_peered":"2024-09-05T13:14:32.410016+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:32.410016+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6909,"log_dups_size":0,"ondisk_log_size":6909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28274688,"num_objects":6903,"num_object_clones":0,"num_object_copies":13806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4418,"num_write_kb":17672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6971","reported_seq":7106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.614234+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:31.614234+0000","last_peered":"2024-09-05T13:14:31.614234+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:31.614234+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3122,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2550},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7025","reported_seq":7207,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.600617+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:31.600617+0000","last_peered":"2024-09-05T13:14:31.600617+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:31.600617+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7025,"log_dups_size":0,"ondisk_log_size":7025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28741632,"num_objects":7017,"num_object_clones":0,"num_object_copies":14034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2924,"num_write_kb":11696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7040","reported_seq":7174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.664710+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:31.664710+0000","last_peered":"2024-09-05T13:14:31.664710+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:31.664710+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7040,"log_dups_size":0,"ondisk_log_size":7040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28819456,"num_objects":7036,"num_object_clones":0,"num_object_copies":14072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3170,"num_write_kb":12680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2470},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7154","reported_seq":7289,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.618369+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:31.618369+0000","last_peered":"2024-09-05T13:14:31.618369+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:31.618369+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7154,"log_dups_size":0,"ondisk_log_size":7154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29254115,"num_objects":7143,"num_object_clones":0,"num_object_copies":14286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3822,"num_write_kb":15288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":458985955,"num_objects":112058,"num_object_clones":0,"num_object_copies":224116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2700,"num_objects_unfound":0,"num_objects_dirty":112058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67857,"num_write_kb":271428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1688,"num_bytes_recovered":6914048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":929030144,"data_stored":928939819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112139,"ondisk_log_size":112139,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579420,"kb_used_data":367244,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93792420,"statfs":{"total":96636764160,"available":96043438080,"internally_reserved":0,"allocated":376057856,"data_stored":375812766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":217247405},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680904,"kb_used_data":406328,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93690936,"statfs":{"total":96636764160,"available":95939518464,"internally_reserved":0,"allocated":416079872,"data_stored":415830705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,0,4,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271720,"kb_used_data":131736,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94100120,"statfs":{"total":96636764160,"available":96358522880,"internally_reserved":0,"allocated":134897664,"data_stored":134569011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":417062912,"data_stored":417059299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":135835648,"data_stored":135752549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376131584,"data_stored":376127971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:36.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:37.220 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 124 20 114744 114724 3.61302 1.86719 0.0715547 0.0216156 2024-09-05T13:14:37.378 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:37.379 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:37.410 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-05T13:14:35.550665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":458985974,"num_objects":112060,"num_object_clones":0,"num_object_copies":224120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2700,"num_objects_unfound":0,"num_objects_dirty":112060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67859,"num_write_kb":271430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1689,"num_bytes_recovered":6914067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112142,"ondisk_log_size":112142,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532044,"kb_used_data":905308,"kb_used_omap":12,"kb_used_meta":626675,"kb_avail":281583476,"statfs":{"total":289910292480,"available":288341479424,"internally_reserved":0,"allocated":927035392,"data_stored":926212482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":641715206},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,6,2,10,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21270528,"num_objects":5193,"num_object_clones":0,"num_object_copies":10386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":5193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5193,"num_write_kb":20772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007275"},"pg_stats":[{"pgid":"2.f","version":"43'6953","reported_seq":7021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.613515+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:31.613515+0000","last_peered":"2024-09-05T13:14:31.613515+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:31.613515+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":6953,"log_dups_size":0,"ondisk_log_size":6953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28479488,"num_objects":6953,"num_object_clones":0,"num_object_copies":13906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6953,"num_write_kb":27812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2532},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6990","reported_seq":7090,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:32.444969+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:32.444969+0000","last_peered":"2024-09-05T13:14:32.444969+0000","last_clean":"2024-09-05T13:14:32.444969+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:32.444969+0000","last_undegraded":"2024-09-05T13:14:32.444969+0000","last_fullsized":"2024-09-05T13:14:32.444969+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":6990,"log_dups_size":0,"ondisk_log_size":6990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28606464,"num_objects":6984,"num_object_clones":0,"num_object_copies":13968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5000,"num_write_kb":20000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7065","reported_seq":7186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:32.436377+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:32.436377+0000","last_peered":"2024-09-05T13:14:32.436377+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:32.436377+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7065,"log_dups_size":0,"ondisk_log_size":7065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28921856,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4876,"num_write_kb":19504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2569}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7066","reported_seq":7131,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:32.469320+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:32.469320+0000","last_peered":"2024-09-05T13:14:32.469320+0000","last_clean":"2024-09-05T13:14:32.469320+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:32.469320+0000","last_undegraded":"2024-09-05T13:14:32.469320+0000","last_fullsized":"2024-09-05T13:14:32.469320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7066,"log_dups_size":0,"ondisk_log_size":7066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28942336,"num_objects":7066,"num_object_clones":0,"num_object_copies":14132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7066,"num_write_kb":28264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7023","reported_seq":7152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:31.630303+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:31.630303+0000","last_peered":"2024-09-05T13:14:31.630303+0000","last_clean":"2024-09-05T13:14:31.630303+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:31.630303+0000","last_undegraded":"2024-09-05T13:14:31.630303+0000","last_fullsized":"2024-09-05T13:14:31.630303+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7023,"log_dups_size":0,"ondisk_log_size":7023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28737536,"num_objects":7016,"num_object_clones":0,"num_object_copies":14032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3563,"num_write_kb":14252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7042","reported_seq":7175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.614524+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:31.614524+0000","last_peered":"2024-09-05T13:14:31.614524+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:31.614524+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7042,"log_dups_size":0,"ondisk_log_size":7042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3497,"num_write_kb":13988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2586},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6942","reported_seq":7076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.519133+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:31.519133+0000","last_peered":"2024-09-05T13:14:31.519133+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:31.519133+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":6942,"log_dups_size":0,"ondisk_log_size":6942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28409856,"num_objects":6936,"num_object_clones":0,"num_object_copies":13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3261,"num_write_kb":13044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2500},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7044","reported_seq":7171,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:31.639913+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:31.639913+0000","last_peered":"2024-09-05T13:14:31.639913+0000","last_clean":"2024-09-05T13:14:31.639913+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:31.639913+0000","last_undegraded":"2024-09-05T13:14:31.639913+0000","last_fullsized":"2024-09-05T13:14:31.639913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7044,"log_dups_size":0,"ondisk_log_size":7044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28844032,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3654,"num_write_kb":14616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6998","reported_seq":7128,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:32.464380+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:32.464380+0000","last_peered":"2024-09-05T13:14:32.464380+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:32.464380+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":6998,"log_dups_size":0,"ondisk_log_size":6998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28635136,"num_objects":6991,"num_object_clones":0,"num_object_copies":13982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":18608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2540}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6948","reported_seq":7081,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.566156+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:31.566156+0000","last_peered":"2024-09-05T13:14:31.566156+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:31.566156+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":6948,"log_dups_size":0,"ondisk_log_size":6948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3315,"num_write_kb":13260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2486},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6969","reported_seq":10451,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:32.439001+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:32.439001+0000","last_peered":"2024-09-05T13:14:32.439001+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:32.439001+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":6969,"log_dups_size":0,"ondisk_log_size":6969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28524544,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2700,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4564,"num_write_kb":18256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1685,"num_bytes_recovered":6901760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2700},{"shards":"2","objects":4262}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6909","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:32.410016+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:32.410016+0000","last_peered":"2024-09-05T13:14:32.410016+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:32.410016+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":6909,"log_dups_size":0,"ondisk_log_size":6909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28274688,"num_objects":6903,"num_object_clones":0,"num_object_copies":13806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4418,"num_write_kb":17672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6971","reported_seq":7106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.614234+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:31.614234+0000","last_peered":"2024-09-05T13:14:31.614234+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:31.614234+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":6971,"log_dups_size":0,"ondisk_log_size":6971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28545024,"num_objects":6969,"num_object_clones":0,"num_object_copies":13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3122,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2550},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7025","reported_seq":7207,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.600617+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:31.600617+0000","last_peered":"2024-09-05T13:14:31.600617+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:31.600617+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7025,"log_dups_size":0,"ondisk_log_size":7025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28741632,"num_objects":7017,"num_object_clones":0,"num_object_copies":14034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2924,"num_write_kb":11696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7040","reported_seq":7174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.664710+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:31.664710+0000","last_peered":"2024-09-05T13:14:31.664710+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:31.664710+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7040,"log_dups_size":0,"ondisk_log_size":7040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28819456,"num_objects":7036,"num_object_clones":0,"num_object_copies":14072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3170,"num_write_kb":12680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2470},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7154","reported_seq":7289,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:31.618369+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:31.618369+0000","last_peered":"2024-09-05T13:14:31.618369+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:31.618369+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7154,"log_dups_size":0,"ondisk_log_size":7154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29254115,"num_objects":7143,"num_object_clones":0,"num_object_copies":14286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3822,"num_write_kb":15288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":458985955,"num_objects":112058,"num_object_clones":0,"num_object_copies":224116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2700,"num_objects_unfound":0,"num_objects_dirty":112058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67857,"num_write_kb":271428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1688,"num_bytes_recovered":6914048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":929030144,"data_stored":928939819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112139,"ondisk_log_size":112139,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579420,"kb_used_data":367244,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93792420,"statfs":{"total":96636764160,"available":96043438080,"internally_reserved":0,"allocated":376057856,"data_stored":375812766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":217247405},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680904,"kb_used_data":406328,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93690936,"statfs":{"total":96636764160,"available":95939518464,"internally_reserved":0,"allocated":416079872,"data_stored":415830705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,0,4,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271720,"kb_used_data":131736,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94100120,"statfs":{"total":96636764160,"available":96358522880,"internally_reserved":0,"allocated":134897664,"data_stored":134569011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":417062912,"data_stored":417059299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":135835648,"data_stored":135752549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376131584,"data_stored":376127971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:37.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:37.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6953", 'reported_seq': 7021, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:31.613515+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:31.613515+0000', 'last_peered': '2024-09-05T13:14:31.613515+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:31.613515+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 6953, 'log_dups_size': 0, 'ondisk_log_size': 6953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28479488, 'num_objects': 6953, 'num_object_clones': 0, 'num_object_copies': 13906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6953, 'num_write_kb': 27812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2532}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:37.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:37.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7065", 'reported_seq': 7186, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:32.436377+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:32.436377+0000', 'last_peered': '2024-09-05T13:14:32.436377+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:32.436377+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 7065, 'log_dups_size': 0, 'ondisk_log_size': 7065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28921856, 'num_objects': 7061, 'num_object_clones': 0, 'num_object_copies': 14122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7061, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4876, 'num_write_kb': 19504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 2569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7042", 'reported_seq': 7175, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:31.614524+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:31.614524+0000', 'last_peered': '2024-09-05T13:14:31.614524+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:31.614524+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 7042, 'log_dups_size': 0, 'ondisk_log_size': 7042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28823552, 'num_objects': 7037, 'num_object_clones': 0, 'num_object_copies': 14074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3497, 'num_write_kb': 13988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2586}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6942", 'reported_seq': 7076, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:31.519133+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:31.519133+0000', 'last_peered': '2024-09-05T13:14:31.519133+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:31.519133+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 6942, 'log_dups_size': 0, 'ondisk_log_size': 6942, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28409856, 'num_objects': 6936, 'num_object_clones': 0, 'num_object_copies': 13872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6936, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3261, 'num_write_kb': 13044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2500}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6998", 'reported_seq': 7128, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:32.464380+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:32.464380+0000', 'last_peered': '2024-09-05T13:14:32.464380+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:32.464380+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 6998, 'log_dups_size': 0, 'ondisk_log_size': 6998, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28635136, 'num_objects': 6991, 'num_object_clones': 0, 'num_object_copies': 13982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6991, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4652, 'num_write_kb': 18608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 2540}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6948", 'reported_seq': 7081, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:31.566156+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:31.566156+0000', 'last_peered': '2024-09-05T13:14:31.566156+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:31.566156+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 6948, 'log_dups_size': 0, 'ondisk_log_size': 6948, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28426240, 'num_objects': 6940, 'num_object_clones': 0, 'num_object_copies': 13880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6940, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3315, 'num_write_kb': 13260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2486}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:37.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6969", 'reported_seq': 10451, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:32.439001+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:32.439001+0000', 'last_peered': '2024-09-05T13:14:32.439001+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:32.439001+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 6969, 'log_dups_size': 0, 'ondisk_log_size': 6969, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28524544, 'num_objects': 6964, 'num_object_clones': 0, 'num_object_copies': 13928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2700, 'num_objects_unfound': 0, 'num_objects_dirty': 6964, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4564, 'num_write_kb': 18256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1685, 'num_bytes_recovered': 6901760, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2700}, {'shards': '2', 'objects': 4262}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6909", 'reported_seq': 7026, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:32.410016+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:32.410016+0000', 'last_peered': '2024-09-05T13:14:32.410016+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:32.410016+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 6909, 'log_dups_size': 0, 'ondisk_log_size': 6909, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28274688, 'num_objects': 6903, 'num_object_clones': 0, 'num_object_copies': 13806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4418, 'num_write_kb': 17672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 2516}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6971", 'reported_seq': 7106, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:31.614234+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:31.614234+0000', 'last_peered': '2024-09-05T13:14:31.614234+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:31.614234+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 6971, 'log_dups_size': 0, 'ondisk_log_size': 6971, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28545024, 'num_objects': 6969, 'num_object_clones': 0, 'num_object_copies': 13938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6969, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3122, 'num_write_kb': 12488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2550}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7025", 'reported_seq': 7207, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:31.600617+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:31.600617+0000', 'last_peered': '2024-09-05T13:14:31.600617+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:31.600617+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 7025, 'log_dups_size': 0, 'ondisk_log_size': 7025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28741632, 'num_objects': 7017, 'num_object_clones': 0, 'num_object_copies': 14034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2924, 'num_write_kb': 11696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2573}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7040", 'reported_seq': 7174, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:31.664710+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:31.664710+0000', 'last_peered': '2024-09-05T13:14:31.664710+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:31.664710+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 7040, 'log_dups_size': 0, 'ondisk_log_size': 7040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28819456, 'num_objects': 7036, 'num_object_clones': 0, 'num_object_copies': 14072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7036, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3170, 'num_write_kb': 12680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2470}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:37.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7154", 'reported_seq': 7289, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:31.618369+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:31.618369+0000', 'last_peered': '2024-09-05T13:14:31.618369+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:31.618369+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 7154, 'log_dups_size': 0, 'ondisk_log_size': 7154, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29254115, 'num_objects': 7143, 'num_object_clones': 0, 'num_object_copies': 14286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7143, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3822, 'num_write_kb': 15288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2578}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:37.414 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:38.107 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:38.108 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/224120 objects degraded (21.911%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":204,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112060,"data_bytes":458985974,"bytes_used":1568813056,"bytes_avail":288341479424,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":224120,"degraded_ratio":0.2191147599500268,"misplaced_objects":2700,"misplaced_total":224120,"misplaced_ratio":0.012047117615563091,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":84940,"recovering_keys_per_sec":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"write_bytes_sec":1771469,"read_op_per_sec":0,"write_op_per_sec":432},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:14:38.133 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:38.133 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:38.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 125 20 115211 115191 3.59863 1.82422 0.03645 0.0217002 2024-09-05T13:14:38.690 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:38.690 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:38.713 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-05T13:14:37.551339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":469533174,"num_objects":114635,"num_object_clones":0,"num_object_copies":229270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":114635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70434,"num_write_kb":281730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1813,"num_bytes_recovered":7421971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114717,"ondisk_log_size":114717,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1559000,"kb_used_data":918376,"kb_used_omap":12,"kb_used_meta":640563,"kb_avail":281556520,"statfs":{"total":289910292480,"available":288313876480,"internally_reserved":0,"allocated":940417024,"data_stored":939594114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":655936518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,7,9,12,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31817728,"num_objects":7768,"num_object_clones":0,"num_object_copies":15536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-373,"num_objects_unfound":0,"num_objects_dirty":7768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":31072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":1527808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007302"},"pg_stats":[{"pgid":"2.f","version":"43'7110","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.754083+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:36.754083+0000","last_peered":"2024-09-05T13:14:36.754083+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:36.754083+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7110,"log_dups_size":0,"ondisk_log_size":7110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29122560,"num_objects":7110,"num_object_clones":0,"num_object_copies":14220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7110,"num_write_kb":28440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2689},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7156","reported_seq":7255,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:37.437477+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:37.437477+0000","last_peered":"2024-09-05T13:14:37.437477+0000","last_clean":"2024-09-05T13:14:37.437477+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:37.437477+0000","last_undegraded":"2024-09-05T13:14:37.437477+0000","last_fullsized":"2024-09-05T13:14:37.437477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29286400,"num_objects":7150,"num_object_clones":0,"num_object_copies":14300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5166,"num_write_kb":20664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7218","reported_seq":7340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.408980+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:37.408980+0000","last_peered":"2024-09-05T13:14:37.408980+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:37.408980+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5029,"num_write_kb":20116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2722}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7228","reported_seq":7293,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:37.260030+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:37.260030+0000","last_peered":"2024-09-05T13:14:37.260030+0000","last_clean":"2024-09-05T13:14:37.260030+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:37.260030+0000","last_undegraded":"2024-09-05T13:14:37.260030+0000","last_fullsized":"2024-09-05T13:14:37.260030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7228,"log_dups_size":0,"ondisk_log_size":7228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7228,"num_write_kb":28912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7186","reported_seq":7315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:36.630322+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:36.630322+0000","last_peered":"2024-09-05T13:14:36.630322+0000","last_clean":"2024-09-05T13:14:36.630322+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:36.630322+0000","last_undegraded":"2024-09-05T13:14:36.630322+0000","last_fullsized":"2024-09-05T13:14:36.630322+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7186,"log_dups_size":0,"ondisk_log_size":7186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3726,"num_write_kb":14904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7192","reported_seq":7325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.697070+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:36.697070+0000","last_peered":"2024-09-05T13:14:36.697070+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:36.697070+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7192,"log_dups_size":0,"ondisk_log_size":7192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29437952,"num_objects":7187,"num_object_clones":0,"num_object_copies":14374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3647,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2736},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7094","reported_seq":7229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.707411+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:36.707411+0000","last_peered":"2024-09-05T13:14:36.707411+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:36.707411+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7094,"log_dups_size":0,"ondisk_log_size":7094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29032448,"num_objects":7088,"num_object_clones":0,"num_object_copies":14176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3413,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2652},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7186","reported_seq":7313,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:36.713373+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:36.713373+0000","last_peered":"2024-09-05T13:14:36.713373+0000","last_clean":"2024-09-05T13:14:36.713373+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:36.713373+0000","last_undegraded":"2024-09-05T13:14:36.713373+0000","last_fullsized":"2024-09-05T13:14:36.713373+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7186,"log_dups_size":0,"ondisk_log_size":7186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29425664,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3796,"num_write_kb":15184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7156","reported_seq":7286,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.403595+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:37.403595+0000","last_peered":"2024-09-05T13:14:37.403595+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:37.403595+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29282304,"num_objects":7149,"num_object_clones":0,"num_object_copies":14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2698}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7109","reported_seq":7241,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.689628+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:36.689628+0000","last_peered":"2024-09-05T13:14:36.689628+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:36.689628+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7109,"log_dups_size":0,"ondisk_log_size":7109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3476,"num_write_kb":13904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2647},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7140","reported_seq":10870,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.470975+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:37.470975+0000","last_peered":"2024-09-05T13:14:37.470975+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:37.470975+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7140,"log_dups_size":0,"ondisk_log_size":7140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29224960,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4735,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1809,"num_bytes_recovered":7409664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2576},{"shards":"2","objects":4557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7062","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.462397+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:37.462397+0000","last_peered":"2024-09-05T13:14:37.462397+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:37.462397+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7062,"log_dups_size":0,"ondisk_log_size":7062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901376,"num_objects":7056,"num_object_clones":0,"num_object_copies":14112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4571,"num_write_kb":18284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7126","reported_seq":7261,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.709335+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:36.709335+0000","last_peered":"2024-09-05T13:14:36.709335+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:36.709335+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7126,"log_dups_size":0,"ondisk_log_size":7126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29179904,"num_objects":7124,"num_object_clones":0,"num_object_copies":14248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3277,"num_write_kb":13108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2705},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7211","reported_seq":7393,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.682867+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:36.682867+0000","last_peered":"2024-09-05T13:14:36.682867+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:36.682867+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7211,"log_dups_size":0,"ondisk_log_size":7211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29503488,"num_objects":7203,"num_object_clones":0,"num_object_copies":14406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3110,"num_write_kb":12440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2759},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7224","reported_seq":7358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.683082+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:36.683082+0000","last_peered":"2024-09-05T13:14:36.683082+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:36.683082+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7224,"log_dups_size":0,"ondisk_log_size":7224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29573120,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3354,"num_write_kb":13416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2654},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7316","reported_seq":7450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.642329+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:36.642329+0000","last_peered":"2024-09-05T13:14:36.642329+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:36.642329+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7316,"log_dups_size":0,"ondisk_log_size":7316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29917667,"num_objects":7305,"num_object_clones":0,"num_object_copies":14610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3984,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2740},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":469533155,"num_objects":114633,"num_object_clones":0,"num_object_copies":229266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":114633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70432,"num_write_kb":281728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7421952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942198784,"data_stored":942108459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114714,"ondisk_log_size":114714,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606896,"kb_used_data":372512,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93764944,"statfs":{"total":96636764160,"available":96015302656,"internally_reserved":0,"allocated":381452288,"data_stored":381207198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":239988397},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680384,"kb_used_data":414128,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93691456,"statfs":{"total":96636764160,"available":95940050944,"internally_reserved":0,"allocated":424067072,"data_stored":423817905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":271720,"kb_used_data":131736,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94100120,"statfs":{"total":96636764160,"available":96358522880,"internally_reserved":0,"allocated":134897664,"data_stored":134569011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425029632,"data_stored":425026019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":135835648,"data_stored":135752549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381333504,"data_stored":381329891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:39.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 126 20 115709 115689 3.58548 1.94531 0.0225971 0.0217803 2024-09-05T13:14:40.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 127 20 116236 116216 3.57346 2.05859 0.0260897 0.0218553 2024-09-05T13:14:41.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 128 20 116769 116749 3.56181 2.08203 0.104447 0.0219266 2024-09-05T13:14:41.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:42.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 129 20 117265 117245 3.54922 1.9375 0.0882415 0.0220018 2024-09-05T13:14:42.265 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:42.287 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-05T13:14:41.552628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":469533174,"num_objects":114635,"num_object_clones":0,"num_object_copies":229270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":114635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70434,"num_write_kb":281730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1813,"num_bytes_recovered":7421971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114717,"ondisk_log_size":114717,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1571180,"kb_used_data":926396,"kb_used_omap":12,"kb_used_meta":644723,"kb_avail":281544340,"statfs":{"total":289910292480,"available":288301404160,"internally_reserved":0,"allocated":948629504,"data_stored":947806594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":660196358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,10,3,10,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21147648,"num_objects":5163,"num_object_clones":0,"num_object_copies":10326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5163,"num_write_kb":20652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007405"},"pg_stats":[{"pgid":"2.f","version":"43'7110","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.754083+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:36.754083+0000","last_peered":"2024-09-05T13:14:36.754083+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:36.754083+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7110,"log_dups_size":0,"ondisk_log_size":7110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29122560,"num_objects":7110,"num_object_clones":0,"num_object_copies":14220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7110,"num_write_kb":28440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2689},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7156","reported_seq":7255,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:37.437477+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:37.437477+0000","last_peered":"2024-09-05T13:14:37.437477+0000","last_clean":"2024-09-05T13:14:37.437477+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:37.437477+0000","last_undegraded":"2024-09-05T13:14:37.437477+0000","last_fullsized":"2024-09-05T13:14:37.437477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29286400,"num_objects":7150,"num_object_clones":0,"num_object_copies":14300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5166,"num_write_kb":20664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7218","reported_seq":7340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.408980+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:37.408980+0000","last_peered":"2024-09-05T13:14:37.408980+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:37.408980+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5029,"num_write_kb":20116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2722}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7228","reported_seq":7293,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:37.260030+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:37.260030+0000","last_peered":"2024-09-05T13:14:37.260030+0000","last_clean":"2024-09-05T13:14:37.260030+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:37.260030+0000","last_undegraded":"2024-09-05T13:14:37.260030+0000","last_fullsized":"2024-09-05T13:14:37.260030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7228,"log_dups_size":0,"ondisk_log_size":7228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7228,"num_write_kb":28912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7186","reported_seq":7315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:36.630322+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:36.630322+0000","last_peered":"2024-09-05T13:14:36.630322+0000","last_clean":"2024-09-05T13:14:36.630322+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:36.630322+0000","last_undegraded":"2024-09-05T13:14:36.630322+0000","last_fullsized":"2024-09-05T13:14:36.630322+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7186,"log_dups_size":0,"ondisk_log_size":7186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3726,"num_write_kb":14904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7192","reported_seq":7325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.697070+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:36.697070+0000","last_peered":"2024-09-05T13:14:36.697070+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:36.697070+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7192,"log_dups_size":0,"ondisk_log_size":7192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29437952,"num_objects":7187,"num_object_clones":0,"num_object_copies":14374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3647,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2736},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7094","reported_seq":7229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.707411+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:36.707411+0000","last_peered":"2024-09-05T13:14:36.707411+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:36.707411+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7094,"log_dups_size":0,"ondisk_log_size":7094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29032448,"num_objects":7088,"num_object_clones":0,"num_object_copies":14176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3413,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2652},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7186","reported_seq":7313,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:36.713373+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:36.713373+0000","last_peered":"2024-09-05T13:14:36.713373+0000","last_clean":"2024-09-05T13:14:36.713373+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:36.713373+0000","last_undegraded":"2024-09-05T13:14:36.713373+0000","last_fullsized":"2024-09-05T13:14:36.713373+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7186,"log_dups_size":0,"ondisk_log_size":7186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29425664,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3796,"num_write_kb":15184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7156","reported_seq":7286,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.403595+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:37.403595+0000","last_peered":"2024-09-05T13:14:37.403595+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:37.403595+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29282304,"num_objects":7149,"num_object_clones":0,"num_object_copies":14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2698}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7109","reported_seq":7241,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.689628+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:36.689628+0000","last_peered":"2024-09-05T13:14:36.689628+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:36.689628+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7109,"log_dups_size":0,"ondisk_log_size":7109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3476,"num_write_kb":13904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2647},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7140","reported_seq":10870,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.470975+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:37.470975+0000","last_peered":"2024-09-05T13:14:37.470975+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:37.470975+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7140,"log_dups_size":0,"ondisk_log_size":7140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29224960,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4735,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1809,"num_bytes_recovered":7409664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2576},{"shards":"2","objects":4557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7062","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.462397+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:37.462397+0000","last_peered":"2024-09-05T13:14:37.462397+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:37.462397+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7062,"log_dups_size":0,"ondisk_log_size":7062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901376,"num_objects":7056,"num_object_clones":0,"num_object_copies":14112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4571,"num_write_kb":18284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7126","reported_seq":7261,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.709335+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:36.709335+0000","last_peered":"2024-09-05T13:14:36.709335+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:36.709335+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7126,"log_dups_size":0,"ondisk_log_size":7126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29179904,"num_objects":7124,"num_object_clones":0,"num_object_copies":14248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3277,"num_write_kb":13108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2705},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7211","reported_seq":7393,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.682867+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:36.682867+0000","last_peered":"2024-09-05T13:14:36.682867+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:36.682867+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7211,"log_dups_size":0,"ondisk_log_size":7211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29503488,"num_objects":7203,"num_object_clones":0,"num_object_copies":14406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3110,"num_write_kb":12440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2759},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7224","reported_seq":7358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.683082+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:36.683082+0000","last_peered":"2024-09-05T13:14:36.683082+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:36.683082+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7224,"log_dups_size":0,"ondisk_log_size":7224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29573120,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3354,"num_write_kb":13416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2654},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7316","reported_seq":7450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.642329+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:36.642329+0000","last_peered":"2024-09-05T13:14:36.642329+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:36.642329+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7316,"log_dups_size":0,"ondisk_log_size":7316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29917667,"num_objects":7305,"num_object_clones":0,"num_object_copies":14610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3984,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2740},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":469533155,"num_objects":114633,"num_object_clones":0,"num_object_copies":229266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":114633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70432,"num_write_kb":281728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7421952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":950579200,"data_stored":950488875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114714,"ondisk_log_size":114714,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606896,"kb_used_data":372512,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93764944,"statfs":{"total":96636764160,"available":96015302656,"internally_reserved":0,"allocated":381452288,"data_stored":381207198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":239988397},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680384,"kb_used_data":414128,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93691456,"statfs":{"total":96636764160,"available":95940050944,"internally_reserved":0,"allocated":424067072,"data_stored":423817905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":283900,"kb_used_data":139756,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94087940,"statfs":{"total":96636764160,"available":96346050560,"internally_reserved":0,"allocated":143110144,"data_stored":142781491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425029632,"data_stored":425026019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":144216064,"data_stored":144132965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381333504,"data_stored":381329891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:42.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:42.844 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:42.871 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-05T13:14:41.552628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":469533174,"num_objects":114635,"num_object_clones":0,"num_object_copies":229270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":114635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70434,"num_write_kb":281730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1813,"num_bytes_recovered":7421971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114717,"ondisk_log_size":114717,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1571180,"kb_used_data":926396,"kb_used_omap":12,"kb_used_meta":644723,"kb_avail":281544340,"statfs":{"total":289910292480,"available":288301404160,"internally_reserved":0,"allocated":948629504,"data_stored":947806594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":660196358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,10,3,10,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21147648,"num_objects":5163,"num_object_clones":0,"num_object_copies":10326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5163,"num_write_kb":20652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007405"},"pg_stats":[{"pgid":"2.f","version":"43'7110","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.754083+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:36.754083+0000","last_peered":"2024-09-05T13:14:36.754083+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:36.754083+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7110,"log_dups_size":0,"ondisk_log_size":7110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29122560,"num_objects":7110,"num_object_clones":0,"num_object_copies":14220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7110,"num_write_kb":28440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2689},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7156","reported_seq":7255,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:37.437477+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:37.437477+0000","last_peered":"2024-09-05T13:14:37.437477+0000","last_clean":"2024-09-05T13:14:37.437477+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:37.437477+0000","last_undegraded":"2024-09-05T13:14:37.437477+0000","last_fullsized":"2024-09-05T13:14:37.437477+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29286400,"num_objects":7150,"num_object_clones":0,"num_object_copies":14300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5166,"num_write_kb":20664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7218","reported_seq":7340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.408980+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:37.408980+0000","last_peered":"2024-09-05T13:14:37.408980+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:37.408980+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7218,"log_dups_size":0,"ondisk_log_size":7218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5029,"num_write_kb":20116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2722}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7228","reported_seq":7293,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:37.260030+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:37.260030+0000","last_peered":"2024-09-05T13:14:37.260030+0000","last_clean":"2024-09-05T13:14:37.260030+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:37.260030+0000","last_undegraded":"2024-09-05T13:14:37.260030+0000","last_fullsized":"2024-09-05T13:14:37.260030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7228,"log_dups_size":0,"ondisk_log_size":7228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7228,"num_write_kb":28912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7186","reported_seq":7315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:36.630322+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:36.630322+0000","last_peered":"2024-09-05T13:14:36.630322+0000","last_clean":"2024-09-05T13:14:36.630322+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:36.630322+0000","last_undegraded":"2024-09-05T13:14:36.630322+0000","last_fullsized":"2024-09-05T13:14:36.630322+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7186,"log_dups_size":0,"ondisk_log_size":7186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29405184,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3726,"num_write_kb":14904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7192","reported_seq":7325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.697070+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:36.697070+0000","last_peered":"2024-09-05T13:14:36.697070+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:36.697070+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7192,"log_dups_size":0,"ondisk_log_size":7192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29437952,"num_objects":7187,"num_object_clones":0,"num_object_copies":14374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3647,"num_write_kb":14588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2736},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7094","reported_seq":7229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.707411+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:36.707411+0000","last_peered":"2024-09-05T13:14:36.707411+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:36.707411+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7094,"log_dups_size":0,"ondisk_log_size":7094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29032448,"num_objects":7088,"num_object_clones":0,"num_object_copies":14176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3413,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2652},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7186","reported_seq":7313,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:36.713373+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:36.713373+0000","last_peered":"2024-09-05T13:14:36.713373+0000","last_clean":"2024-09-05T13:14:36.713373+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:36.713373+0000","last_undegraded":"2024-09-05T13:14:36.713373+0000","last_fullsized":"2024-09-05T13:14:36.713373+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7186,"log_dups_size":0,"ondisk_log_size":7186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29425664,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3796,"num_write_kb":15184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7156","reported_seq":7286,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.403595+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:37.403595+0000","last_peered":"2024-09-05T13:14:37.403595+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:37.403595+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7156,"log_dups_size":0,"ondisk_log_size":7156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29282304,"num_objects":7149,"num_object_clones":0,"num_object_copies":14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2698}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7109","reported_seq":7241,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.689628+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:36.689628+0000","last_peered":"2024-09-05T13:14:36.689628+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:36.689628+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7109,"log_dups_size":0,"ondisk_log_size":7109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3476,"num_write_kb":13904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2647},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7140","reported_seq":10870,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.470975+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:37.470975+0000","last_peered":"2024-09-05T13:14:37.470975+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:37.470975+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7140,"log_dups_size":0,"ondisk_log_size":7140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29224960,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4735,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1809,"num_bytes_recovered":7409664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2576},{"shards":"2","objects":4557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7062","reported_seq":7180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:37.462397+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:37.462397+0000","last_peered":"2024-09-05T13:14:37.462397+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:37.462397+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7062,"log_dups_size":0,"ondisk_log_size":7062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901376,"num_objects":7056,"num_object_clones":0,"num_object_copies":14112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4571,"num_write_kb":18284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7126","reported_seq":7261,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.709335+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:36.709335+0000","last_peered":"2024-09-05T13:14:36.709335+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:36.709335+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7126,"log_dups_size":0,"ondisk_log_size":7126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29179904,"num_objects":7124,"num_object_clones":0,"num_object_copies":14248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3277,"num_write_kb":13108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2705},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7211","reported_seq":7393,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.682867+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:36.682867+0000","last_peered":"2024-09-05T13:14:36.682867+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:36.682867+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7211,"log_dups_size":0,"ondisk_log_size":7211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29503488,"num_objects":7203,"num_object_clones":0,"num_object_copies":14406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3110,"num_write_kb":12440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2759},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7224","reported_seq":7358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.683082+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:36.683082+0000","last_peered":"2024-09-05T13:14:36.683082+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:36.683082+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7224,"log_dups_size":0,"ondisk_log_size":7224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29573120,"num_objects":7220,"num_object_clones":0,"num_object_copies":14440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3354,"num_write_kb":13416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2654},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7316","reported_seq":7450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:36.642329+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:36.642329+0000","last_peered":"2024-09-05T13:14:36.642329+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:36.642329+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7316,"log_dups_size":0,"ondisk_log_size":7316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29917667,"num_objects":7305,"num_object_clones":0,"num_object_copies":14610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3984,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2740},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":469533155,"num_objects":114633,"num_object_clones":0,"num_object_copies":229266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2576,"num_objects_unfound":0,"num_objects_dirty":114633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70432,"num_write_kb":281728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1812,"num_bytes_recovered":7421952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":950579200,"data_stored":950488875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114714,"ondisk_log_size":114714,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606896,"kb_used_data":372512,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93764944,"statfs":{"total":96636764160,"available":96015302656,"internally_reserved":0,"allocated":381452288,"data_stored":381207198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":239988397},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680384,"kb_used_data":414128,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93691456,"statfs":{"total":96636764160,"available":95940050944,"internally_reserved":0,"allocated":424067072,"data_stored":423817905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":283900,"kb_used_data":139756,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94087940,"statfs":{"total":96636764160,"available":96346050560,"internally_reserved":0,"allocated":143110144,"data_stored":142781491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425029632,"data_stored":425026019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":144216064,"data_stored":144132965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381333504,"data_stored":381329891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:42.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:42.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7110", 'reported_seq': 7180, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:36.754083+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:36.754083+0000', 'last_peered': '2024-09-05T13:14:36.754083+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:36.754083+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 7110, 'log_dups_size': 0, 'ondisk_log_size': 7110, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29122560, 'num_objects': 7110, 'num_object_clones': 0, 'num_object_copies': 14220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7110, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7110, 'num_write_kb': 28440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2689}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:42.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7218", 'reported_seq': 7340, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:37.408980+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:37.408980+0000', 'last_peered': '2024-09-05T13:14:37.408980+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:37.408980+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 7218, 'log_dups_size': 0, 'ondisk_log_size': 7218, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29548544, 'num_objects': 7214, 'num_object_clones': 0, 'num_object_copies': 14428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7214, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5029, 'num_write_kb': 20116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 2722}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7192", 'reported_seq': 7325, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:36.697070+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:36.697070+0000', 'last_peered': '2024-09-05T13:14:36.697070+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:36.697070+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 7192, 'log_dups_size': 0, 'ondisk_log_size': 7192, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29437952, 'num_objects': 7187, 'num_object_clones': 0, 'num_object_copies': 14374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7187, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3647, 'num_write_kb': 14588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2736}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7094", 'reported_seq': 7229, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:36.707411+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:36.707411+0000', 'last_peered': '2024-09-05T13:14:36.707411+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:36.707411+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 7094, 'log_dups_size': 0, 'ondisk_log_size': 7094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29032448, 'num_objects': 7088, 'num_object_clones': 0, 'num_object_copies': 14176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7088, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3413, 'num_write_kb': 13652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2652}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7156", 'reported_seq': 7286, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:37.403595+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:37.403595+0000', 'last_peered': '2024-09-05T13:14:37.403595+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:37.403595+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 7156, 'log_dups_size': 0, 'ondisk_log_size': 7156, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29282304, 'num_objects': 7149, 'num_object_clones': 0, 'num_object_copies': 14298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4810, 'num_write_kb': 19240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 2698}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7109", 'reported_seq': 7241, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:36.689628+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:36.689628+0000', 'last_peered': '2024-09-05T13:14:36.689628+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:36.689628+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 7109, 'log_dups_size': 0, 'ondisk_log_size': 7109, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29085696, 'num_objects': 7101, 'num_object_clones': 0, 'num_object_copies': 14202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3476, 'num_write_kb': 13904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2647}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:42.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7140", 'reported_seq': 10870, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:37.470975+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:37.470975+0000', 'last_peered': '2024-09-05T13:14:37.470975+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:37.470975+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 7140, 'log_dups_size': 0, 'ondisk_log_size': 7140, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29224960, 'num_objects': 7135, 'num_object_clones': 0, 'num_object_copies': 14270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2576, 'num_objects_unfound': 0, 'num_objects_dirty': 7135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4735, 'num_write_kb': 18940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1809, 'num_bytes_recovered': 7409664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2576}, {'shards': '2', 'objects': 4557}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7062", 'reported_seq': 7180, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:37.462397+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:37.462397+0000', 'last_peered': '2024-09-05T13:14:37.462397+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:37.462397+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 7062, 'log_dups_size': 0, 'ondisk_log_size': 7062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28901376, 'num_objects': 7056, 'num_object_clones': 0, 'num_object_copies': 14112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4571, 'num_write_kb': 18284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 2669}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7126", 'reported_seq': 7261, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:36.709335+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:36.709335+0000', 'last_peered': '2024-09-05T13:14:36.709335+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:36.709335+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 7126, 'log_dups_size': 0, 'ondisk_log_size': 7126, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29179904, 'num_objects': 7124, 'num_object_clones': 0, 'num_object_copies': 14248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3277, 'num_write_kb': 13108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2705}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7211", 'reported_seq': 7393, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:36.682867+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:36.682867+0000', 'last_peered': '2024-09-05T13:14:36.682867+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:36.682867+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 7211, 'log_dups_size': 0, 'ondisk_log_size': 7211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29503488, 'num_objects': 7203, 'num_object_clones': 0, 'num_object_copies': 14406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3110, 'num_write_kb': 12440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2759}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7224", 'reported_seq': 7358, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:36.683082+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:36.683082+0000', 'last_peered': '2024-09-05T13:14:36.683082+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:36.683082+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 7224, 'log_dups_size': 0, 'ondisk_log_size': 7224, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29573120, 'num_objects': 7220, 'num_object_clones': 0, 'num_object_copies': 14440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3354, 'num_write_kb': 13416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2654}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:42.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7316", 'reported_seq': 7450, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:36.642329+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:36.642329+0000', 'last_peered': '2024-09-05T13:14:36.642329+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:36.642329+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 7316, 'log_dups_size': 0, 'ondisk_log_size': 7316, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29917667, 'num_objects': 7305, 'num_object_clones': 0, 'num_object_copies': 14610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3984, 'num_write_kb': 15936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2740}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:42.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:43.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 130 20 117713 117693 3.53538 1.75 0.026326 0.0220887 2024-09-05T13:14:43.552 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:43.552 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/229270 objects degraded (21.419%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":210,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":114635,"data_bytes":469533174,"bytes_used":1608888320,"bytes_avail":288301404160,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":229270,"degraded_ratio":0.21419287303179657,"misplaced_objects":2576,"misplaced_total":229270,"misplaced_ratio":0.011235661011035025,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83233,"recovering_keys_per_sec":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"write_bytes_sec":1761217,"read_op_per_sec":0,"write_op_per_sec":429},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:14:43.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:43.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:44.119 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:44.120 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:44.151 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-05T13:14:43.553241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":479924726,"num_objects":117172,"num_object_clones":0,"num_object_copies":234344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":117172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72971,"num_write_kb":291878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1941,"num_bytes_recovered":7946259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117254,"ondisk_log_size":117254,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1591956,"kb_used_data":938852,"kb_used_omap":12,"kb_used_meta":653043,"kb_avail":281523564,"statfs":{"total":289910292480,"available":288280129536,"internally_reserved":0,"allocated":961384448,"data_stored":960561538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":668716038},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,6,7,5,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31539200,"num_objects":7700,"num_object_clones":0,"num_object_copies":15400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-372,"num_objects_unfound":0,"num_objects_dirty":7700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7700,"num_write_kb":30800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004178"},"pg_stats":[{"pgid":"2.f","version":"43'7268","reported_seq":7337,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:41.731152+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:41.731152+0000","last_peered":"2024-09-05T13:14:41.731152+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:41.731152+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7268,"log_dups_size":0,"ondisk_log_size":7268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29769728,"num_objects":7268,"num_object_clones":0,"num_object_copies":14536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":29072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2847},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7307","reported_seq":7406,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:42.450643+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:42.450643+0000","last_peered":"2024-09-05T13:14:42.450643+0000","last_clean":"2024-09-05T13:14:42.450643+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:42.450643+0000","last_undegraded":"2024-09-05T13:14:42.450643+0000","last_fullsized":"2024-09-05T13:14:42.450643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7307,"log_dups_size":0,"ondisk_log_size":7307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29904896,"num_objects":7301,"num_object_clones":0,"num_object_copies":14602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5317,"num_write_kb":21268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7404","reported_seq":7527,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:42.486016+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:42.486016+0000","last_peered":"2024-09-05T13:14:42.486016+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:42.486016+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7404,"log_dups_size":0,"ondisk_log_size":7404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30310400,"num_objects":7400,"num_object_clones":0,"num_object_copies":14800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5215,"num_write_kb":20860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":2908}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7363","reported_seq":7428,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:42.348723+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:42.348723+0000","last_peered":"2024-09-05T13:14:42.348723+0000","last_clean":"2024-09-05T13:14:42.348723+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:42.348723+0000","last_undegraded":"2024-09-05T13:14:42.348723+0000","last_fullsized":"2024-09-05T13:14:42.348723+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7363,"log_dups_size":0,"ondisk_log_size":7363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30158848,"num_objects":7363,"num_object_clones":0,"num_object_copies":14726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7363,"num_write_kb":29452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7336","reported_seq":7465,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:41.760443+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:41.760443+0000","last_peered":"2024-09-05T13:14:41.760443+0000","last_clean":"2024-09-05T13:14:41.760443+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:41.760443+0000","last_undegraded":"2024-09-05T13:14:41.760443+0000","last_fullsized":"2024-09-05T13:14:41.760443+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7336,"log_dups_size":0,"ondisk_log_size":7336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3876,"num_write_kb":15504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7356","reported_seq":7487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:41.770549+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:41.770549+0000","last_peered":"2024-09-05T13:14:41.770549+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:41.770549+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30109696,"num_objects":7351,"num_object_clones":0,"num_object_copies":14702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3811,"num_write_kb":15244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2900},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7235","reported_seq":7370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:41.705956+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:41.705956+0000","last_peered":"2024-09-05T13:14:41.705956+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:41.705956+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7235,"log_dups_size":0,"ondisk_log_size":7235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29609984,"num_objects":7229,"num_object_clones":0,"num_object_copies":14458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":14216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7353","reported_seq":7480,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:41.740710+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:41.740710+0000","last_peered":"2024-09-05T13:14:41.740710+0000","last_clean":"2024-09-05T13:14:41.740710+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:41.740710+0000","last_undegraded":"2024-09-05T13:14:41.740710+0000","last_fullsized":"2024-09-05T13:14:41.740710+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7353,"log_dups_size":0,"ondisk_log_size":7353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30109696,"num_objects":7351,"num_object_clones":0,"num_object_copies":14702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3963,"num_write_kb":15852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7304","reported_seq":7434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:42.487104+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:42.487104+0000","last_peered":"2024-09-05T13:14:42.487104+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:42.487104+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7304,"log_dups_size":0,"ondisk_log_size":7304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29888512,"num_objects":7297,"num_object_clones":0,"num_object_copies":14594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4958,"num_write_kb":19832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":2846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7276","reported_seq":7409,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:41.716582+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:41.716582+0000","last_peered":"2024-09-05T13:14:41.716582+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:41.716582+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7276,"log_dups_size":0,"ondisk_log_size":7276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29769728,"num_objects":7268,"num_object_clones":0,"num_object_copies":14536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3643,"num_write_kb":14572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2814},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7305","reported_seq":11291,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:42.429977+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:42.429977+0000","last_peered":"2024-09-05T13:14:42.429977+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:42.429977+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7305,"log_dups_size":0,"ondisk_log_size":7305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4900,"num_write_kb":19600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1937,"num_bytes_recovered":7933952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2448},{"shards":"2","objects":4850}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7201","reported_seq":7318,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:42.421667+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:42.421667+0000","last_peered":"2024-09-05T13:14:42.421667+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:42.421667+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7201,"log_dups_size":0,"ondisk_log_size":7201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29470720,"num_objects":7195,"num_object_clones":0,"num_object_copies":14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4710,"num_write_kb":18840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2808}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7300","reported_seq":7434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:41.767842+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:41.767842+0000","last_peered":"2024-09-05T13:14:41.767842+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:41.767842+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7300,"log_dups_size":0,"ondisk_log_size":7300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3451,"num_write_kb":13804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2879},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7375","reported_seq":7557,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:41.769927+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:41.769927+0000","last_peered":"2024-09-05T13:14:41.769927+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:41.769927+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7375,"log_dups_size":0,"ondisk_log_size":7375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30175232,"num_objects":7367,"num_object_clones":0,"num_object_copies":14734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3274,"num_write_kb":13096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2923},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7396","reported_seq":7530,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:41.787656+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:41.787656+0000","last_peered":"2024-09-05T13:14:41.787656+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:41.787656+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7396,"log_dups_size":0,"ondisk_log_size":7396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3526,"num_write_kb":14104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2826},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7472","reported_seq":7607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:41.754608+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:41.754608+0000","last_peered":"2024-09-05T13:14:41.754608+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:41.754608+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7472,"log_dups_size":0,"ondisk_log_size":7472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30556643,"num_objects":7461,"num_object_clones":0,"num_object_copies":14922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4140,"num_write_kb":16560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2896},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":479924707,"num_objects":117170,"num_object_clones":0,"num_object_copies":234340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":117170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72969,"num_write_kb":291876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1940,"num_bytes_recovered":7946240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":963399680,"data_stored":963309355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117251,"ondisk_log_size":117251,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616064,"kb_used_data":377520,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93755776,"statfs":{"total":96636764160,"available":96005914624,"internally_reserved":0,"allocated":386580480,"data_stored":386335390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691992,"kb_used_data":421576,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93679848,"statfs":{"total":96636764160,"available":95928164352,"internally_reserved":0,"allocated":431693824,"data_stored":431444657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":283900,"kb_used_data":139756,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94087940,"statfs":{"total":96636764160,"available":96346050560,"internally_reserved":0,"allocated":143110144,"data_stored":142781491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432795648,"data_stored":432792035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":144216064,"data_stored":144132965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386387968,"data_stored":386384355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:44.227 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 131 20 118193 118173 3.52264 1.875 0.016548 0.0221714 2024-09-05T13:14:45.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 132 20 118715 118695 3.51135 2.03906 0.0257459 0.0222428 2024-09-05T13:14:46.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 133 20 119247 119227 3.50058 2.07812 0.0180135 0.0223091 2024-09-05T13:14:47.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:47.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 134 20 119775 119755 3.48985 2.0625 0.0124654 0.0223768 2024-09-05T13:14:47.700 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:47.702 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:47.729 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-05T13:14:47.554693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":490299894,"num_objects":119705,"num_object_clones":0,"num_object_copies":239410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":119705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75504,"num_write_kb":302010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":8441875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119787,"ondisk_log_size":119787,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1614160,"kb_used_data":959648,"kb_used_omap":12,"kb_used_meta":654451,"kb_avail":281501360,"statfs":{"total":289910292480,"available":288257392640,"internally_reserved":0,"allocated":982679552,"data_stored":981856642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":670157830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,7,10,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31313920,"num_objects":7645,"num_object_clones":0,"num_object_copies":15290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-373,"num_objects_unfound":0,"num_objects_dirty":7645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7645,"num_write_kb":30580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":1527808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004027"},"pg_stats":[{"pgid":"2.f","version":"43'7414","reported_seq":7484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.747950+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:46.747950+0000","last_peered":"2024-09-05T13:14:46.747950+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:46.747950+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7414,"num_write_kb":29656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2993},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7474","reported_seq":7574,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:47.466074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:47.466074+0000","last_peered":"2024-09-05T13:14:47.466074+0000","last_clean":"2024-09-05T13:14:47.466074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:47.466074+0000","last_undegraded":"2024-09-05T13:14:47.466074+0000","last_fullsized":"2024-09-05T13:14:47.466074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7474,"log_dups_size":0,"ondisk_log_size":7474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30588928,"num_objects":7468,"num_object_clones":0,"num_object_copies":14936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":21936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7566","reported_seq":7687,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.490075+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:47.490075+0000","last_peered":"2024-09-05T13:14:47.490075+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:47.490075+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30973952,"num_objects":7562,"num_object_clones":0,"num_object_copies":15124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5377,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7513","reported_seq":7578,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:47.328072+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:47.328072+0000","last_peered":"2024-09-05T13:14:47.328072+0000","last_clean":"2024-09-05T13:14:47.328072+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:47.328072+0000","last_undegraded":"2024-09-05T13:14:47.328072+0000","last_fullsized":"2024-09-05T13:14:47.328072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7513,"log_dups_size":0,"ondisk_log_size":7513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30773248,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7485","reported_seq":7614,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:46.714249+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:46.714249+0000","last_peered":"2024-09-05T13:14:46.714249+0000","last_clean":"2024-09-05T13:14:46.714249+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:46.714249+0000","last_undegraded":"2024-09-05T13:14:46.714249+0000","last_fullsized":"2024-09-05T13:14:46.714249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7485,"log_dups_size":0,"ondisk_log_size":7485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30629888,"num_objects":7478,"num_object_clones":0,"num_object_copies":14956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7536","reported_seq":7669,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.694003+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:46.694003+0000","last_peered":"2024-09-05T13:14:46.694003+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:46.694003+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7536,"log_dups_size":0,"ondisk_log_size":7536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30846976,"num_objects":7531,"num_object_clones":0,"num_object_copies":15062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3991,"num_write_kb":15964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3080},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7394","reported_seq":7528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.849715+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:46.849715+0000","last_peered":"2024-09-05T13:14:46.849715+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:46.849715+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3713,"num_write_kb":14852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7513","reported_seq":7640,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:46.809252+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:46.809252+0000","last_peered":"2024-09-05T13:14:46.809252+0000","last_clean":"2024-09-05T13:14:46.809252+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:46.809252+0000","last_undegraded":"2024-09-05T13:14:46.809252+0000","last_fullsized":"2024-09-05T13:14:46.809252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7513,"log_dups_size":0,"ondisk_log_size":7513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4123,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7460","reported_seq":7590,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.490077+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:47.490077+0000","last_peered":"2024-09-05T13:14:47.490077+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:47.490077+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7460,"log_dups_size":0,"ondisk_log_size":7460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30527488,"num_objects":7453,"num_object_clones":0,"num_object_copies":14906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7449","reported_seq":7582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.817467+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:46.817467+0000","last_peered":"2024-09-05T13:14:46.817467+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:46.817467+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3816,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2987},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7477","reported_seq":11705,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.500172+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:47.500172+0000","last_peered":"2024-09-05T13:14:47.500172+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:47.500172+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7477,"log_dups_size":0,"ondisk_log_size":7477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":8429568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2327},{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7385","reported_seq":7501,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.477990+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:47.477990+0000","last_peered":"2024-09-05T13:14:47.477990+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:47.477990+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4894,"num_write_kb":19576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2992}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7448","reported_seq":7583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.845309+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:46.845309+0000","last_peered":"2024-09-05T13:14:46.845309+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:46.845309+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7448,"log_dups_size":0,"ondisk_log_size":7448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3599,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3027},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7516","reported_seq":7698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.821292+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:46.821292+0000","last_peered":"2024-09-05T13:14:46.821292+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:46.821292+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7516,"log_dups_size":0,"ondisk_log_size":7516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":13660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3064},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7538","reported_seq":7672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.817151+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:46.817151+0000","last_peered":"2024-09-05T13:14:46.817151+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:46.817151+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7538,"log_dups_size":0,"ondisk_log_size":7538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3668,"num_write_kb":14672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7616","reported_seq":7751,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.785880+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:46.785880+0000","last_peered":"2024-09-05T13:14:46.785880+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:46.785880+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7616,"log_dups_size":0,"ondisk_log_size":7616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146467,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4284,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":490299875,"num_objects":119703,"num_object_clones":0,"num_object_copies":239406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":119703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75502,"num_write_kb":302008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2061,"num_bytes_recovered":8441856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":984731648,"data_stored":984641323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119784,"ondisk_log_size":119784,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608688,"kb_used_data":382624,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93763152,"statfs":{"total":96636764160,"available":96013467648,"internally_reserved":0,"allocated":391806976,"data_stored":391561886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":231468717},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703648,"kb_used_data":429072,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93668192,"statfs":{"total":96636764160,"available":95916228608,"internally_reserved":0,"allocated":439369728,"data_stored":439120561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301824,"kb_used_data":147952,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94070016,"statfs":{"total":96636764160,"available":96327696384,"internally_reserved":0,"allocated":151502848,"data_stored":151174195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":157544111},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440422400,"data_stored":440418787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152584192,"data_stored":152501093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391725056,"data_stored":391721443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:47.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:48.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 135 20 120234 120214 3.47728 1.79297 0.0176605 0.0224568 2024-09-05T13:14:48.307 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:48.307 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:48.329 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-05T13:14:47.554693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":490299894,"num_objects":119705,"num_object_clones":0,"num_object_copies":239410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":119705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75504,"num_write_kb":302010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":8441875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119787,"ondisk_log_size":119787,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1614160,"kb_used_data":959648,"kb_used_omap":12,"kb_used_meta":654451,"kb_avail":281501360,"statfs":{"total":289910292480,"available":288257392640,"internally_reserved":0,"allocated":982679552,"data_stored":981856642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":670157830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,7,10,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31313920,"num_objects":7645,"num_object_clones":0,"num_object_copies":15290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-373,"num_objects_unfound":0,"num_objects_dirty":7645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7645,"num_write_kb":30580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":1527808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004027"},"pg_stats":[{"pgid":"2.f","version":"43'7414","reported_seq":7484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.747950+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:46.747950+0000","last_peered":"2024-09-05T13:14:46.747950+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:46.747950+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7414,"num_write_kb":29656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2993},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7474","reported_seq":7574,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:47.466074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:47.466074+0000","last_peered":"2024-09-05T13:14:47.466074+0000","last_clean":"2024-09-05T13:14:47.466074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:47.466074+0000","last_undegraded":"2024-09-05T13:14:47.466074+0000","last_fullsized":"2024-09-05T13:14:47.466074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7474,"log_dups_size":0,"ondisk_log_size":7474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30588928,"num_objects":7468,"num_object_clones":0,"num_object_copies":14936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":21936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7566","reported_seq":7687,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.490075+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:47.490075+0000","last_peered":"2024-09-05T13:14:47.490075+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:47.490075+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30973952,"num_objects":7562,"num_object_clones":0,"num_object_copies":15124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5377,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7513","reported_seq":7578,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:47.328072+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:47.328072+0000","last_peered":"2024-09-05T13:14:47.328072+0000","last_clean":"2024-09-05T13:14:47.328072+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:47.328072+0000","last_undegraded":"2024-09-05T13:14:47.328072+0000","last_fullsized":"2024-09-05T13:14:47.328072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7513,"log_dups_size":0,"ondisk_log_size":7513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30773248,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7485","reported_seq":7614,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:46.714249+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:46.714249+0000","last_peered":"2024-09-05T13:14:46.714249+0000","last_clean":"2024-09-05T13:14:46.714249+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:46.714249+0000","last_undegraded":"2024-09-05T13:14:46.714249+0000","last_fullsized":"2024-09-05T13:14:46.714249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7485,"log_dups_size":0,"ondisk_log_size":7485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30629888,"num_objects":7478,"num_object_clones":0,"num_object_copies":14956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7536","reported_seq":7669,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.694003+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:46.694003+0000","last_peered":"2024-09-05T13:14:46.694003+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:46.694003+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7536,"log_dups_size":0,"ondisk_log_size":7536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30846976,"num_objects":7531,"num_object_clones":0,"num_object_copies":15062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3991,"num_write_kb":15964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3080},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7394","reported_seq":7528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.849715+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:46.849715+0000","last_peered":"2024-09-05T13:14:46.849715+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:46.849715+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3713,"num_write_kb":14852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7513","reported_seq":7640,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:46.809252+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:46.809252+0000","last_peered":"2024-09-05T13:14:46.809252+0000","last_clean":"2024-09-05T13:14:46.809252+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:46.809252+0000","last_undegraded":"2024-09-05T13:14:46.809252+0000","last_fullsized":"2024-09-05T13:14:46.809252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7513,"log_dups_size":0,"ondisk_log_size":7513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4123,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7460","reported_seq":7590,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.490077+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:47.490077+0000","last_peered":"2024-09-05T13:14:47.490077+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:47.490077+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7460,"log_dups_size":0,"ondisk_log_size":7460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30527488,"num_objects":7453,"num_object_clones":0,"num_object_copies":14906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7449","reported_seq":7582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.817467+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:46.817467+0000","last_peered":"2024-09-05T13:14:46.817467+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:46.817467+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3816,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2987},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7477","reported_seq":11705,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.500172+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:47.500172+0000","last_peered":"2024-09-05T13:14:47.500172+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:47.500172+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7477,"log_dups_size":0,"ondisk_log_size":7477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":8429568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2327},{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7385","reported_seq":7501,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.477990+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:47.477990+0000","last_peered":"2024-09-05T13:14:47.477990+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:47.477990+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4894,"num_write_kb":19576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2992}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7448","reported_seq":7583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.845309+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:46.845309+0000","last_peered":"2024-09-05T13:14:46.845309+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:46.845309+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7448,"log_dups_size":0,"ondisk_log_size":7448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3599,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3027},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7516","reported_seq":7698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.821292+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:46.821292+0000","last_peered":"2024-09-05T13:14:46.821292+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:46.821292+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7516,"log_dups_size":0,"ondisk_log_size":7516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":13660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3064},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7538","reported_seq":7672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.817151+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:46.817151+0000","last_peered":"2024-09-05T13:14:46.817151+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:46.817151+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7538,"log_dups_size":0,"ondisk_log_size":7538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3668,"num_write_kb":14672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7616","reported_seq":7751,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.785880+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:46.785880+0000","last_peered":"2024-09-05T13:14:46.785880+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:46.785880+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7616,"log_dups_size":0,"ondisk_log_size":7616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146467,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4284,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":490299875,"num_objects":119703,"num_object_clones":0,"num_object_copies":239406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":119703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75502,"num_write_kb":302008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2061,"num_bytes_recovered":8441856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":984731648,"data_stored":984641323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119784,"ondisk_log_size":119784,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608688,"kb_used_data":382624,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93763152,"statfs":{"total":96636764160,"available":96013467648,"internally_reserved":0,"allocated":391806976,"data_stored":391561886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":231468717},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703648,"kb_used_data":429072,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93668192,"statfs":{"total":96636764160,"available":95916228608,"internally_reserved":0,"allocated":439369728,"data_stored":439120561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301824,"kb_used_data":147952,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94070016,"statfs":{"total":96636764160,"available":96327696384,"internally_reserved":0,"allocated":151502848,"data_stored":151174195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":157544111},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440422400,"data_stored":440418787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152584192,"data_stored":152501093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391725056,"data_stored":391721443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:48.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:48.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7414", 'reported_seq': 7484, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:46.747950+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:46.747950+0000', 'last_peered': '2024-09-05T13:14:46.747950+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:46.747950+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 7414, 'log_dups_size': 0, 'ondisk_log_size': 7414, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30367744, 'num_objects': 7414, 'num_object_clones': 0, 'num_object_copies': 14828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7414, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7414, 'num_write_kb': 29656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2993}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:48.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:48.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7566", 'reported_seq': 7687, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:47.490075+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:47.490075+0000', 'last_peered': '2024-09-05T13:14:47.490075+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:47.490075+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 7566, 'log_dups_size': 0, 'ondisk_log_size': 7566, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30973952, 'num_objects': 7562, 'num_object_clones': 0, 'num_object_copies': 15124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7562, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5377, 'num_write_kb': 21508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 3070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:48.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7536", 'reported_seq': 7669, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:46.694003+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:46.694003+0000', 'last_peered': '2024-09-05T13:14:46.694003+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:46.694003+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 7536, 'log_dups_size': 0, 'ondisk_log_size': 7536, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30846976, 'num_objects': 7531, 'num_object_clones': 0, 'num_object_copies': 15062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7531, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3991, 'num_write_kb': 15964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3080}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7394", 'reported_seq': 7528, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:46.849715+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:46.849715+0000', 'last_peered': '2024-09-05T13:14:46.849715+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:46.849715+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 7394, 'log_dups_size': 0, 'ondisk_log_size': 7394, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30261248, 'num_objects': 7388, 'num_object_clones': 0, 'num_object_copies': 14776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3713, 'num_write_kb': 14852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2952}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7460", 'reported_seq': 7590, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:47.490077+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:47.490077+0000', 'last_peered': '2024-09-05T13:14:47.490077+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:47.490077+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 7460, 'log_dups_size': 0, 'ondisk_log_size': 7460, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30527488, 'num_objects': 7453, 'num_object_clones': 0, 'num_object_copies': 14906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7453, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5114, 'num_write_kb': 20456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 3002}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7449", 'reported_seq': 7582, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:46.817467+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:46.817467+0000', 'last_peered': '2024-09-05T13:14:46.817467+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:46.817467+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 7449, 'log_dups_size': 0, 'ondisk_log_size': 7449, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30478336, 'num_objects': 7441, 'num_object_clones': 0, 'num_object_copies': 14882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7441, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3816, 'num_write_kb': 15264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2987}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7477", 'reported_seq': 11705, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:47.500172+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:47.500172+0000', 'last_peered': '2024-09-05T13:14:47.500172+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:47.500172+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 7477, 'log_dups_size': 0, 'ondisk_log_size': 7477, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30605312, 'num_objects': 7472, 'num_object_clones': 0, 'num_object_copies': 14944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2327, 'num_objects_unfound': 0, 'num_objects_dirty': 7472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5072, 'num_write_kb': 20288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2058, 'num_bytes_recovered': 8429568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2327}, {'shards': '2', 'objects': 5143}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:48.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7385", 'reported_seq': 7501, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:47.477990+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:47.477990+0000', 'last_peered': '2024-09-05T13:14:47.477990+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:47.477990+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 7385, 'log_dups_size': 0, 'ondisk_log_size': 7385, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30224384, 'num_objects': 7379, 'num_object_clones': 0, 'num_object_copies': 14758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4894, 'num_write_kb': 19576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 2992}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7448", 'reported_seq': 7583, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:46.845309+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:46.845309+0000', 'last_peered': '2024-09-05T13:14:46.845309+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:46.845309+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 7448, 'log_dups_size': 0, 'ondisk_log_size': 7448, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30498816, 'num_objects': 7446, 'num_object_clones': 0, 'num_object_copies': 14892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3599, 'num_write_kb': 14396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3027}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7516", 'reported_seq': 7698, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:46.821292+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:46.821292+0000', 'last_peered': '2024-09-05T13:14:46.821292+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:46.821292+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 7516, 'log_dups_size': 0, 'ondisk_log_size': 7516, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30752768, 'num_objects': 7508, 'num_object_clones': 0, 'num_object_copies': 15016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3415, 'num_write_kb': 13660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3064}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7538", 'reported_seq': 7672, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:46.817151+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:46.817151+0000', 'last_peered': '2024-09-05T13:14:46.817151+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:46.817151+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 7538, 'log_dups_size': 0, 'ondisk_log_size': 7538, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30859264, 'num_objects': 7534, 'num_object_clones': 0, 'num_object_copies': 15068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3668, 'num_write_kb': 14672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2968}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:48.332 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7616", 'reported_seq': 7751, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:46.785880+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:46.785880+0000', 'last_peered': '2024-09-05T13:14:46.785880+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:46.785880+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 7616, 'log_dups_size': 0, 'ondisk_log_size': 7616, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31146467, 'num_objects': 7605, 'num_object_clones': 0, 'num_object_copies': 15210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4284, 'num_write_kb': 17136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3040}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:48.333 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:49.015 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:49.015 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/239410 objects degraded (20.512%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":215,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119705,"data_bytes":490299894,"bytes_used":1652899840,"bytes_avail":288257392640,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":239410,"degraded_ratio":0.2051209222672403,"misplaced_objects":2327,"misplaced_total":239410,"misplaced_ratio":0.0097197276638402736,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":127274,"recovering_keys_per_sec":0,"num_objects_recovered":373,"num_bytes_recovered":1527808,"num_keys_recovered":0,"write_bytes_sec":2608617,"read_op_per_sec":0,"write_op_per_sec":636},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:14:49.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:49.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:49.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 136 20 120692 120672 3.46487 1.78906 0.0402168 0.02254 2024-09-05T13:14:49.617 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:49.618 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:49.641 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-05T13:14:49.555408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":490299894,"num_objects":119705,"num_object_clones":0,"num_object_copies":239410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":119705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75504,"num_write_kb":302010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":8441875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119787,"ondisk_log_size":119787,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1630384,"kb_used_data":967552,"kb_used_omap":12,"kb_used_meta":662771,"kb_avail":281485136,"statfs":{"total":289910292480,"available":288240779264,"internally_reserved":0,"allocated":990773248,"data_stored":989950338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":678677510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,8,10,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":20280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004069"},"pg_stats":[{"pgid":"2.f","version":"43'7414","reported_seq":7484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.747950+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:46.747950+0000","last_peered":"2024-09-05T13:14:46.747950+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:46.747950+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7414,"num_write_kb":29656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2993},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7474","reported_seq":7574,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:47.466074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:47.466074+0000","last_peered":"2024-09-05T13:14:47.466074+0000","last_clean":"2024-09-05T13:14:47.466074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:47.466074+0000","last_undegraded":"2024-09-05T13:14:47.466074+0000","last_fullsized":"2024-09-05T13:14:47.466074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7474,"log_dups_size":0,"ondisk_log_size":7474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30588928,"num_objects":7468,"num_object_clones":0,"num_object_copies":14936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":21936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7566","reported_seq":7687,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.490075+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:47.490075+0000","last_peered":"2024-09-05T13:14:47.490075+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:47.490075+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30973952,"num_objects":7562,"num_object_clones":0,"num_object_copies":15124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5377,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7513","reported_seq":7578,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:47.328072+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:47.328072+0000","last_peered":"2024-09-05T13:14:47.328072+0000","last_clean":"2024-09-05T13:14:47.328072+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:47.328072+0000","last_undegraded":"2024-09-05T13:14:47.328072+0000","last_fullsized":"2024-09-05T13:14:47.328072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7513,"log_dups_size":0,"ondisk_log_size":7513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30773248,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7485","reported_seq":7614,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:46.714249+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:46.714249+0000","last_peered":"2024-09-05T13:14:46.714249+0000","last_clean":"2024-09-05T13:14:46.714249+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:46.714249+0000","last_undegraded":"2024-09-05T13:14:46.714249+0000","last_fullsized":"2024-09-05T13:14:46.714249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7485,"log_dups_size":0,"ondisk_log_size":7485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30629888,"num_objects":7478,"num_object_clones":0,"num_object_copies":14956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7536","reported_seq":7669,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.694003+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:46.694003+0000","last_peered":"2024-09-05T13:14:46.694003+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:46.694003+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7536,"log_dups_size":0,"ondisk_log_size":7536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30846976,"num_objects":7531,"num_object_clones":0,"num_object_copies":15062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3991,"num_write_kb":15964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3080},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7394","reported_seq":7528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.849715+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:46.849715+0000","last_peered":"2024-09-05T13:14:46.849715+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:46.849715+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3713,"num_write_kb":14852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7513","reported_seq":7640,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:46.809252+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:46.809252+0000","last_peered":"2024-09-05T13:14:46.809252+0000","last_clean":"2024-09-05T13:14:46.809252+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:46.809252+0000","last_undegraded":"2024-09-05T13:14:46.809252+0000","last_fullsized":"2024-09-05T13:14:46.809252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7513,"log_dups_size":0,"ondisk_log_size":7513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4123,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7460","reported_seq":7590,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.490077+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:47.490077+0000","last_peered":"2024-09-05T13:14:47.490077+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:47.490077+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7460,"log_dups_size":0,"ondisk_log_size":7460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30527488,"num_objects":7453,"num_object_clones":0,"num_object_copies":14906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7449","reported_seq":7582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.817467+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:46.817467+0000","last_peered":"2024-09-05T13:14:46.817467+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:46.817467+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3816,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2987},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7477","reported_seq":11705,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.500172+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:47.500172+0000","last_peered":"2024-09-05T13:14:47.500172+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:47.500172+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7477,"log_dups_size":0,"ondisk_log_size":7477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":8429568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2327},{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7385","reported_seq":7501,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.477990+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:47.477990+0000","last_peered":"2024-09-05T13:14:47.477990+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:47.477990+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4894,"num_write_kb":19576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2992}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7448","reported_seq":7583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.845309+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:46.845309+0000","last_peered":"2024-09-05T13:14:46.845309+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:46.845309+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7448,"log_dups_size":0,"ondisk_log_size":7448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3599,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3027},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7516","reported_seq":7698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.821292+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:46.821292+0000","last_peered":"2024-09-05T13:14:46.821292+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:46.821292+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7516,"log_dups_size":0,"ondisk_log_size":7516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":13660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3064},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7538","reported_seq":7672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.817151+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:46.817151+0000","last_peered":"2024-09-05T13:14:46.817151+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:46.817151+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7538,"log_dups_size":0,"ondisk_log_size":7538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3668,"num_write_kb":14672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7616","reported_seq":7751,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.785880+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:46.785880+0000","last_peered":"2024-09-05T13:14:46.785880+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:46.785880+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7616,"log_dups_size":0,"ondisk_log_size":7616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146467,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4284,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":490299875,"num_objects":119703,"num_object_clones":0,"num_object_copies":239406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":119703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75502,"num_write_kb":302008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2061,"num_bytes_recovered":8441856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":992903168,"data_stored":992812843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119784,"ondisk_log_size":119784,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608688,"kb_used_data":382624,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93763152,"statfs":{"total":96636764160,"available":96013467648,"internally_reserved":0,"allocated":391806976,"data_stored":391561886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":231468717},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703648,"kb_used_data":429072,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93668192,"statfs":{"total":96636764160,"available":95916228608,"internally_reserved":0,"allocated":439369728,"data_stored":439120561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318048,"kb_used_data":155856,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94053792,"statfs":{"total":96636764160,"available":96311083008,"internally_reserved":0,"allocated":159596544,"data_stored":159267891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":166063791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440422400,"data_stored":440418787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160755712,"data_stored":160672613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391725056,"data_stored":391721443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:50.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 137 20 121182 121162 3.45355 1.91406 0.0143389 0.0226127 2024-09-05T13:14:51.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 138 20 121704 121684 3.4433 2.03906 0.0500947 0.0226817 2024-09-05T13:14:52.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 139 20 122227 122207 3.43323 2.04297 0.0950217 0.0227468 2024-09-05T13:14:52.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:53.192 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:53.216 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-05T13:14:51.555946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":490299894,"num_objects":119705,"num_object_clones":0,"num_object_copies":239410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":119705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75504,"num_write_kb":302010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2062,"num_bytes_recovered":8441875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119787,"ondisk_log_size":119787,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1630384,"kb_used_data":967552,"kb_used_omap":12,"kb_used_meta":662771,"kb_avail":281485136,"statfs":{"total":289910292480,"available":288240779264,"internally_reserved":0,"allocated":990773248,"data_stored":989950338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":678677510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,8,10,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":20280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003945"},"pg_stats":[{"pgid":"2.f","version":"43'7414","reported_seq":7484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.747950+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:46.747950+0000","last_peered":"2024-09-05T13:14:46.747950+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:46.747950+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367744,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7414,"num_write_kb":29656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2993},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7474","reported_seq":7574,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:47.466074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:47.466074+0000","last_peered":"2024-09-05T13:14:47.466074+0000","last_clean":"2024-09-05T13:14:47.466074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:47.466074+0000","last_undegraded":"2024-09-05T13:14:47.466074+0000","last_fullsized":"2024-09-05T13:14:47.466074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7474,"log_dups_size":0,"ondisk_log_size":7474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30588928,"num_objects":7468,"num_object_clones":0,"num_object_copies":14936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":21936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7566","reported_seq":7687,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.490075+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:47.490075+0000","last_peered":"2024-09-05T13:14:47.490075+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:47.490075+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30973952,"num_objects":7562,"num_object_clones":0,"num_object_copies":15124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5377,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7513","reported_seq":7578,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:47.328072+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:47.328072+0000","last_peered":"2024-09-05T13:14:47.328072+0000","last_clean":"2024-09-05T13:14:47.328072+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:47.328072+0000","last_undegraded":"2024-09-05T13:14:47.328072+0000","last_fullsized":"2024-09-05T13:14:47.328072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7513,"log_dups_size":0,"ondisk_log_size":7513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30773248,"num_objects":7513,"num_object_clones":0,"num_object_copies":15026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7485","reported_seq":7614,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:46.714249+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:46.714249+0000","last_peered":"2024-09-05T13:14:46.714249+0000","last_clean":"2024-09-05T13:14:46.714249+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:46.714249+0000","last_undegraded":"2024-09-05T13:14:46.714249+0000","last_fullsized":"2024-09-05T13:14:46.714249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7485,"log_dups_size":0,"ondisk_log_size":7485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30629888,"num_objects":7478,"num_object_clones":0,"num_object_copies":14956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7536","reported_seq":7669,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.694003+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:46.694003+0000","last_peered":"2024-09-05T13:14:46.694003+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:46.694003+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7536,"log_dups_size":0,"ondisk_log_size":7536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30846976,"num_objects":7531,"num_object_clones":0,"num_object_copies":15062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3991,"num_write_kb":15964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3080},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7394","reported_seq":7528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.849715+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:46.849715+0000","last_peered":"2024-09-05T13:14:46.849715+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:46.849715+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3713,"num_write_kb":14852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2952},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7513","reported_seq":7640,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:46.809252+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:46.809252+0000","last_peered":"2024-09-05T13:14:46.809252+0000","last_clean":"2024-09-05T13:14:46.809252+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:46.809252+0000","last_undegraded":"2024-09-05T13:14:46.809252+0000","last_fullsized":"2024-09-05T13:14:46.809252+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7513,"log_dups_size":0,"ondisk_log_size":7513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4123,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7460","reported_seq":7590,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.490077+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:47.490077+0000","last_peered":"2024-09-05T13:14:47.490077+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:47.490077+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7460,"log_dups_size":0,"ondisk_log_size":7460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30527488,"num_objects":7453,"num_object_clones":0,"num_object_copies":14906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3002}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7449","reported_seq":7582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.817467+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:46.817467+0000","last_peered":"2024-09-05T13:14:46.817467+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:46.817467+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7449,"log_dups_size":0,"ondisk_log_size":7449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3816,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2987},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7477","reported_seq":11705,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.500172+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:47.500172+0000","last_peered":"2024-09-05T13:14:47.500172+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:47.500172+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7477,"log_dups_size":0,"ondisk_log_size":7477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":8429568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2327},{"shards":"2","objects":5143}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7385","reported_seq":7501,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:47.477990+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:47.477990+0000","last_peered":"2024-09-05T13:14:47.477990+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:47.477990+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4894,"num_write_kb":19576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":2992}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7448","reported_seq":7583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.845309+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:46.845309+0000","last_peered":"2024-09-05T13:14:46.845309+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:46.845309+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7448,"log_dups_size":0,"ondisk_log_size":7448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3599,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3027},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7516","reported_seq":7698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.821292+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:46.821292+0000","last_peered":"2024-09-05T13:14:46.821292+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:46.821292+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7516,"log_dups_size":0,"ondisk_log_size":7516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":13660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3064},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7538","reported_seq":7672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.817151+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:46.817151+0000","last_peered":"2024-09-05T13:14:46.817151+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:46.817151+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7538,"log_dups_size":0,"ondisk_log_size":7538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3668,"num_write_kb":14672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2968},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7616","reported_seq":7751,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:46.785880+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:46.785880+0000","last_peered":"2024-09-05T13:14:46.785880+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:46.785880+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7616,"log_dups_size":0,"ondisk_log_size":7616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146467,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4284,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3040},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":490299875,"num_objects":119703,"num_object_clones":0,"num_object_copies":239406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2327,"num_objects_unfound":0,"num_objects_dirty":119703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75502,"num_write_kb":302008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2061,"num_bytes_recovered":8441856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":992903168,"data_stored":992812843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119784,"ondisk_log_size":119784,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608688,"kb_used_data":382624,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93763152,"statfs":{"total":96636764160,"available":96013467648,"internally_reserved":0,"allocated":391806976,"data_stored":391561886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":231468717},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703648,"kb_used_data":429072,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93668192,"statfs":{"total":96636764160,"available":95916228608,"internally_reserved":0,"allocated":439369728,"data_stored":439120561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318048,"kb_used_data":155856,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94053792,"statfs":{"total":96636764160,"available":96311083008,"internally_reserved":0,"allocated":159596544,"data_stored":159267891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":166063791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440422400,"data_stored":440418787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160755712,"data_stored":160672613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391725056,"data_stored":391721443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:53.218 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:53.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:14:53.229497+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0228224 2024-09-05T13:14:53.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:14:53.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 140 20 122707 122687 3.4221 1.875 0.0530934 0.0228224 2024-09-05T13:14:53.774 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:53.799 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-05T13:14:53.556558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":500376054,"num_objects":122165,"num_object_clones":0,"num_object_copies":244330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":122165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77964,"num_write_kb":311850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2184,"num_bytes_recovered":8941587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122247,"ondisk_log_size":122247,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1634096,"kb_used_data":979584,"kb_used_omap":12,"kb_used_meta":654451,"kb_avail":281481424,"statfs":{"total":289910292480,"available":288236978176,"internally_reserved":0,"allocated":1003094016,"data_stored":1002271106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":670157830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,5,9,10,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30842880,"num_objects":7530,"num_object_clones":0,"num_object_copies":15060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-371,"num_objects_unfound":0,"num_objects_dirty":7530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7530,"num_write_kb":30120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":371,"num_bytes_recovered":1519616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003929"},"pg_stats":[{"pgid":"2.f","version":"43'7553","reported_seq":7623,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.859335+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:51.859335+0000","last_peered":"2024-09-05T13:14:51.859335+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:51.859335+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7553,"log_dups_size":0,"ondisk_log_size":7553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30937088,"num_objects":7553,"num_object_clones":0,"num_object_copies":15106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7553,"num_write_kb":30212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3132},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7626","reported_seq":7725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:52.543954+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:52.543954+0000","last_peered":"2024-09-05T13:14:52.543954+0000","last_clean":"2024-09-05T13:14:52.543954+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:52.543954+0000","last_undegraded":"2024-09-05T13:14:52.543954+0000","last_fullsized":"2024-09-05T13:14:52.543954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7626,"log_dups_size":0,"ondisk_log_size":7626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31211520,"num_objects":7620,"num_object_clones":0,"num_object_copies":15240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5636,"num_write_kb":22544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7713","reported_seq":7836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.445937+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:52.445937+0000","last_peered":"2024-09-05T13:14:52.445937+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:52.445937+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5524,"num_write_kb":22096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3217}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7663","reported_seq":7728,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:52.531423+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:52.531423+0000","last_peered":"2024-09-05T13:14:52.531423+0000","last_clean":"2024-09-05T13:14:52.531423+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:52.531423+0000","last_undegraded":"2024-09-05T13:14:52.531423+0000","last_fullsized":"2024-09-05T13:14:52.531423+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7663,"log_dups_size":0,"ondisk_log_size":7663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31387648,"num_objects":7663,"num_object_clones":0,"num_object_copies":15326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7663,"num_write_kb":30652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7627","reported_seq":7756,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:51.871788+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:51.871788+0000","last_peered":"2024-09-05T13:14:51.871788+0000","last_clean":"2024-09-05T13:14:51.871788+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:51.871788+0000","last_undegraded":"2024-09-05T13:14:51.871788+0000","last_fullsized":"2024-09-05T13:14:51.871788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7627,"log_dups_size":0,"ondisk_log_size":7627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31211520,"num_objects":7620,"num_object_clones":0,"num_object_copies":15240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7719","reported_seq":7852,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.823150+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:51.823150+0000","last_peered":"2024-09-05T13:14:51.823150+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:51.823150+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7719,"log_dups_size":0,"ondisk_log_size":7719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31596544,"num_objects":7714,"num_object_clones":0,"num_object_copies":15428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4174,"num_write_kb":16696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3263},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7563","reported_seq":7698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.871833+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:51.871833+0000","last_peered":"2024-09-05T13:14:51.871833+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:51.871833+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7563,"log_dups_size":0,"ondisk_log_size":7563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30953472,"num_objects":7557,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3882,"num_write_kb":15528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3121},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7667","reported_seq":7794,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:51.836343+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:51.836343+0000","last_peered":"2024-09-05T13:14:51.836343+0000","last_clean":"2024-09-05T13:14:51.836343+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:51.836343+0000","last_undegraded":"2024-09-05T13:14:51.836343+0000","last_fullsized":"2024-09-05T13:14:51.836343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7667,"log_dups_size":0,"ondisk_log_size":7667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4277,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7607","reported_seq":7737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.451297+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:52.451297+0000","last_peered":"2024-09-05T13:14:52.451297+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:52.451297+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7593","reported_seq":7726,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.786976+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:51.786976+0000","last_peered":"2024-09-05T13:14:51.786976+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:51.786976+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7593,"log_dups_size":0,"ondisk_log_size":7593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31068160,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3960,"num_write_kb":15840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7631","reported_seq":12100,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.513725+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:52.513725+0000","last_peered":"2024-09-05T13:14:52.513725+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:52.513725+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7631,"log_dups_size":0,"ondisk_log_size":7631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31236096,"num_objects":7626,"num_object_clones":0,"num_object_copies":15252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":7626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5226,"num_write_kb":20904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2180,"num_bytes_recovered":8929280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2205},{"shards":"2","objects":5419}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7541","reported_seq":7659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.539945+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:52.539945+0000","last_peered":"2024-09-05T13:14:52.539945+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:52.539945+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5050,"num_write_kb":20200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3148}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7599","reported_seq":7734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.767787+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:51.767787+0000","last_peered":"2024-09-05T13:14:51.767787+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:51.767787+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7599,"log_dups_size":0,"ondisk_log_size":7599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3750,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3178},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7677","reported_seq":7859,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.823514+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:51.823514+0000","last_peered":"2024-09-05T13:14:51.823514+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:51.823514+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7677,"log_dups_size":0,"ondisk_log_size":7677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412224,"num_objects":7669,"num_object_clones":0,"num_object_copies":15338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3576,"num_write_kb":14304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7701","reported_seq":7835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.856978+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:51.856978+0000","last_peered":"2024-09-05T13:14:51.856978+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:51.856978+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7701,"log_dups_size":0,"ondisk_log_size":7701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31526912,"num_objects":7697,"num_object_clones":0,"num_object_copies":15394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3831,"num_write_kb":15324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7764","reported_seq":7899,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.760163+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:51.760163+0000","last_peered":"2024-09-05T13:14:51.760163+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:51.760163+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7764,"log_dups_size":0,"ondisk_log_size":7764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31752675,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3188},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":500376035,"num_objects":122163,"num_object_clones":0,"num_object_copies":244326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":122163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77962,"num_write_kb":311848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":8941568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1005432832,"data_stored":1005342507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122244,"ondisk_log_size":122244,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617536,"kb_used_data":387312,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93754304,"statfs":{"total":96636764160,"available":96004407296,"internally_reserved":0,"allocated":396607488,"data_stored":396362398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":235728557},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698512,"kb_used_data":436416,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93673328,"statfs":{"total":96636764160,"available":95921487872,"internally_reserved":0,"allocated":446889984,"data_stored":446640817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318048,"kb_used_data":155856,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94053792,"statfs":{"total":96636764160,"available":96311083008,"internally_reserved":0,"allocated":159596544,"data_stored":159267891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":166063791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447987712,"data_stored":447984099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160755712,"data_stored":160672613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396689408,"data_stored":396685795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:53.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:53.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7553", 'reported_seq': 7623, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:51.859335+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:51.859335+0000', 'last_peered': '2024-09-05T13:14:51.859335+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:51.859335+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 7553, 'log_dups_size': 0, 'ondisk_log_size': 7553, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30937088, 'num_objects': 7553, 'num_object_clones': 0, 'num_object_copies': 15106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7553, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7553, 'num_write_kb': 30212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3132}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:53.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:53.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7713", 'reported_seq': 7836, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:52.445937+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:52.445937+0000', 'last_peered': '2024-09-05T13:14:52.445937+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:52.445937+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 7713, 'log_dups_size': 0, 'ondisk_log_size': 7713, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31576064, 'num_objects': 7709, 'num_object_clones': 0, 'num_object_copies': 15418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5524, 'num_write_kb': 22096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 3217}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:53.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:53.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7719", 'reported_seq': 7852, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:51.823150+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:51.823150+0000', 'last_peered': '2024-09-05T13:14:51.823150+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:51.823150+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 7719, 'log_dups_size': 0, 'ondisk_log_size': 7719, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31596544, 'num_objects': 7714, 'num_object_clones': 0, 'num_object_copies': 15428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7714, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4174, 'num_write_kb': 16696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3263}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:53.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:53.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7563", 'reported_seq': 7698, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:51.871833+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:51.871833+0000', 'last_peered': '2024-09-05T13:14:51.871833+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:51.871833+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 7563, 'log_dups_size': 0, 'ondisk_log_size': 7563, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30953472, 'num_objects': 7557, 'num_object_clones': 0, 'num_object_copies': 15114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7557, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3882, 'num_write_kb': 15528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3121}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7607", 'reported_seq': 7737, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:52.451297+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:52.451297+0000', 'last_peered': '2024-09-05T13:14:52.451297+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:52.451297+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 7607, 'log_dups_size': 0, 'ondisk_log_size': 7607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31129600, 'num_objects': 7600, 'num_object_clones': 0, 'num_object_copies': 15200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5261, 'num_write_kb': 21044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 3149}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7593", 'reported_seq': 7726, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:51.786976+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:51.786976+0000', 'last_peered': '2024-09-05T13:14:51.786976+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:51.786976+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 7593, 'log_dups_size': 0, 'ondisk_log_size': 7593, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31068160, 'num_objects': 7585, 'num_object_clones': 0, 'num_object_copies': 15170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3960, 'num_write_kb': 15840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3131}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7631", 'reported_seq': 12100, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:52.513725+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:52.513725+0000', 'last_peered': '2024-09-05T13:14:52.513725+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:52.513725+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 7631, 'log_dups_size': 0, 'ondisk_log_size': 7631, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31236096, 'num_objects': 7626, 'num_object_clones': 0, 'num_object_copies': 15252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2205, 'num_objects_unfound': 0, 'num_objects_dirty': 7626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5226, 'num_write_kb': 20904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2180, 'num_bytes_recovered': 8929280, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2205}, {'shards': '2', 'objects': 5419}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7541", 'reported_seq': 7659, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:52.539945+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:52.539945+0000', 'last_peered': '2024-09-05T13:14:52.539945+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:52.539945+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 7541, 'log_dups_size': 0, 'ondisk_log_size': 7541, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30863360, 'num_objects': 7535, 'num_object_clones': 0, 'num_object_copies': 15070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5050, 'num_write_kb': 20200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 3148}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:53.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:53.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7599", 'reported_seq': 7734, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:51.767787+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:51.767787+0000', 'last_peered': '2024-09-05T13:14:51.767787+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:51.767787+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 7599, 'log_dups_size': 0, 'ondisk_log_size': 7599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31117312, 'num_objects': 7597, 'num_object_clones': 0, 'num_object_copies': 15194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3750, 'num_write_kb': 15000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3178}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:53.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:53.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7677", 'reported_seq': 7859, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:51.823514+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:51.823514+0000', 'last_peered': '2024-09-05T13:14:51.823514+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:51.823514+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 7677, 'log_dups_size': 0, 'ondisk_log_size': 7677, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31412224, 'num_objects': 7669, 'num_object_clones': 0, 'num_object_copies': 15338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7669, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3576, 'num_write_kb': 14304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3225}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:53.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:53.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7701", 'reported_seq': 7835, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:51.856978+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:51.856978+0000', 'last_peered': '2024-09-05T13:14:51.856978+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:51.856978+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 7701, 'log_dups_size': 0, 'ondisk_log_size': 7701, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31526912, 'num_objects': 7697, 'num_object_clones': 0, 'num_object_copies': 15394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3831, 'num_write_kb': 15324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3131}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:53.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:53.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7764", 'reported_seq': 7899, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:51.760163+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:51.760163+0000', 'last_peered': '2024-09-05T13:14:51.760163+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:51.760163+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 7764, 'log_dups_size': 0, 'ondisk_log_size': 7764, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31752675, 'num_objects': 7753, 'num_object_clones': 0, 'num_object_copies': 15506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4432, 'num_write_kb': 17728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3188}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:53.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:14:54.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 141 20 123165 123145 3.41052 1.78906 0.0489467 0.0228993 2024-09-05T13:14:54.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:54.562 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/244330 objects degraded (20.099%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":221,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122165,"data_bytes":500376054,"bytes_used":1673314304,"bytes_avail":288236978176,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":244330,"degraded_ratio":0.20099046371710391,"misplaced_objects":2205,"misplaced_total":244330,"misplaced_ratio":0.0090246797364220525,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":126593,"recovering_keys_per_sec":0,"num_objects_recovered":371,"num_bytes_recovered":1519616,"num_keys_recovered":0,"write_bytes_sec":2569398,"read_op_per_sec":0,"write_op_per_sec":627},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:14:54.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:14:54.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:55.119 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:55.119 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:55.143 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-05T13:14:53.556558+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":500376054,"num_objects":122165,"num_object_clones":0,"num_object_copies":244330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":122165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77964,"num_write_kb":311850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2184,"num_bytes_recovered":8941587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122247,"ondisk_log_size":122247,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1634096,"kb_used_data":979584,"kb_used_omap":12,"kb_used_meta":654451,"kb_avail":281481424,"statfs":{"total":289910292480,"available":288236978176,"internally_reserved":0,"allocated":1003094016,"data_stored":1002271106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":670157830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,5,9,10,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30842880,"num_objects":7530,"num_object_clones":0,"num_object_copies":15060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-371,"num_objects_unfound":0,"num_objects_dirty":7530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7530,"num_write_kb":30120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":371,"num_bytes_recovered":1519616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003929"},"pg_stats":[{"pgid":"2.f","version":"43'7553","reported_seq":7623,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.859335+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:51.859335+0000","last_peered":"2024-09-05T13:14:51.859335+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:51.859335+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7553,"log_dups_size":0,"ondisk_log_size":7553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30937088,"num_objects":7553,"num_object_clones":0,"num_object_copies":15106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7553,"num_write_kb":30212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3132},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7626","reported_seq":7725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:52.543954+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:52.543954+0000","last_peered":"2024-09-05T13:14:52.543954+0000","last_clean":"2024-09-05T13:14:52.543954+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:52.543954+0000","last_undegraded":"2024-09-05T13:14:52.543954+0000","last_fullsized":"2024-09-05T13:14:52.543954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7626,"log_dups_size":0,"ondisk_log_size":7626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31211520,"num_objects":7620,"num_object_clones":0,"num_object_copies":15240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5636,"num_write_kb":22544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7713","reported_seq":7836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.445937+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:52.445937+0000","last_peered":"2024-09-05T13:14:52.445937+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:52.445937+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5524,"num_write_kb":22096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3217}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7663","reported_seq":7728,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:52.531423+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:52.531423+0000","last_peered":"2024-09-05T13:14:52.531423+0000","last_clean":"2024-09-05T13:14:52.531423+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:52.531423+0000","last_undegraded":"2024-09-05T13:14:52.531423+0000","last_fullsized":"2024-09-05T13:14:52.531423+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7663,"log_dups_size":0,"ondisk_log_size":7663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31387648,"num_objects":7663,"num_object_clones":0,"num_object_copies":15326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7663,"num_write_kb":30652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7627","reported_seq":7756,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:51.871788+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:51.871788+0000","last_peered":"2024-09-05T13:14:51.871788+0000","last_clean":"2024-09-05T13:14:51.871788+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:51.871788+0000","last_undegraded":"2024-09-05T13:14:51.871788+0000","last_fullsized":"2024-09-05T13:14:51.871788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7627,"log_dups_size":0,"ondisk_log_size":7627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31211520,"num_objects":7620,"num_object_clones":0,"num_object_copies":15240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7719","reported_seq":7852,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.823150+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:51.823150+0000","last_peered":"2024-09-05T13:14:51.823150+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:51.823150+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7719,"log_dups_size":0,"ondisk_log_size":7719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31596544,"num_objects":7714,"num_object_clones":0,"num_object_copies":15428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4174,"num_write_kb":16696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3263},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7563","reported_seq":7698,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.871833+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:51.871833+0000","last_peered":"2024-09-05T13:14:51.871833+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:51.871833+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7563,"log_dups_size":0,"ondisk_log_size":7563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30953472,"num_objects":7557,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3882,"num_write_kb":15528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3121},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7667","reported_seq":7794,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:51.836343+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:51.836343+0000","last_peered":"2024-09-05T13:14:51.836343+0000","last_clean":"2024-09-05T13:14:51.836343+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:51.836343+0000","last_undegraded":"2024-09-05T13:14:51.836343+0000","last_fullsized":"2024-09-05T13:14:51.836343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7667,"log_dups_size":0,"ondisk_log_size":7667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31395840,"num_objects":7665,"num_object_clones":0,"num_object_copies":15330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4277,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7607","reported_seq":7737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.451297+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:52.451297+0000","last_peered":"2024-09-05T13:14:52.451297+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:52.451297+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7593","reported_seq":7726,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.786976+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:51.786976+0000","last_peered":"2024-09-05T13:14:51.786976+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:51.786976+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7593,"log_dups_size":0,"ondisk_log_size":7593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31068160,"num_objects":7585,"num_object_clones":0,"num_object_copies":15170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3960,"num_write_kb":15840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7631","reported_seq":12100,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.513725+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:52.513725+0000","last_peered":"2024-09-05T13:14:52.513725+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:52.513725+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7631,"log_dups_size":0,"ondisk_log_size":7631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31236096,"num_objects":7626,"num_object_clones":0,"num_object_copies":15252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":7626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5226,"num_write_kb":20904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2180,"num_bytes_recovered":8929280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2205},{"shards":"2","objects":5419}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7541","reported_seq":7659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.539945+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:52.539945+0000","last_peered":"2024-09-05T13:14:52.539945+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:52.539945+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5050,"num_write_kb":20200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3148}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7599","reported_seq":7734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.767787+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:51.767787+0000","last_peered":"2024-09-05T13:14:51.767787+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:51.767787+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7599,"log_dups_size":0,"ondisk_log_size":7599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31117312,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3750,"num_write_kb":15000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3178},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7677","reported_seq":7859,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.823514+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:51.823514+0000","last_peered":"2024-09-05T13:14:51.823514+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:51.823514+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7677,"log_dups_size":0,"ondisk_log_size":7677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412224,"num_objects":7669,"num_object_clones":0,"num_object_copies":15338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3576,"num_write_kb":14304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7701","reported_seq":7835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.856978+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:51.856978+0000","last_peered":"2024-09-05T13:14:51.856978+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:51.856978+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7701,"log_dups_size":0,"ondisk_log_size":7701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31526912,"num_objects":7697,"num_object_clones":0,"num_object_copies":15394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3831,"num_write_kb":15324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3131},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7764","reported_seq":7899,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:51.760163+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:51.760163+0000","last_peered":"2024-09-05T13:14:51.760163+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:51.760163+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7764,"log_dups_size":0,"ondisk_log_size":7764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31752675,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3188},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":500376035,"num_objects":122163,"num_object_clones":0,"num_object_copies":244326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":122163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":77962,"num_write_kb":311848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":8941568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1005432832,"data_stored":1005342507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122244,"ondisk_log_size":122244,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617536,"kb_used_data":387312,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93754304,"statfs":{"total":96636764160,"available":96004407296,"internally_reserved":0,"allocated":396607488,"data_stored":396362398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":235728557},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698512,"kb_used_data":436416,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93673328,"statfs":{"total":96636764160,"available":95921487872,"internally_reserved":0,"allocated":446889984,"data_stored":446640817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318048,"kb_used_data":155856,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94053792,"statfs":{"total":96636764160,"available":96311083008,"internally_reserved":0,"allocated":159596544,"data_stored":159267891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":166063791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447987712,"data_stored":447984099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160755712,"data_stored":160672613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396689408,"data_stored":396685795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:55.230 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 142 20 123609 123589 3.39872 1.73438 0.0377872 0.0229781 2024-09-05T13:14:56.230 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 143 20 124123 124103 3.38899 2.00781 0.0350231 0.0230438 2024-09-05T13:14:57.230 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 144 20 124640 124620 3.37948 2.01953 0.0379926 0.0231109 2024-09-05T13:14:58.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:58.230 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 145 20 125148 125128 3.36986 1.98438 0.0215959 0.0231758 2024-09-05T13:14:58.725 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:58.728 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:58.752 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-05T13:14:57.558249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506733046,"num_objects":123717,"num_object_clones":0,"num_object_copies":247434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":123717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79516,"num_write_kb":318058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2184,"num_bytes_recovered":8941587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123799,"ondisk_log_size":123799,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1663420,"kb_used_data":996428,"kb_used_omap":12,"kb_used_meta":666931,"kb_avail":281452100,"statfs":{"total":289910292480,"available":288206950400,"internally_reserved":0,"allocated":1020342272,"data_stored":1019519362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":682937350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,8,4,8,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26808320,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6545,"num_write_kb":26180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004265"},"pg_stats":[{"pgid":"2.f","version":"43'7720","reported_seq":7789,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.875703+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:56.875703+0000","last_peered":"2024-09-05T13:14:56.875703+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:56.875703+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7720,"log_dups_size":0,"ondisk_log_size":7720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31621120,"num_objects":7720,"num_object_clones":0,"num_object_copies":15440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7720,"num_write_kb":30880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3299},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7626","reported_seq":7725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:52.543954+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:52.543954+0000","last_peered":"2024-09-05T13:14:52.543954+0000","last_clean":"2024-09-05T13:14:52.543954+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:52.543954+0000","last_undegraded":"2024-09-05T13:14:52.543954+0000","last_fullsized":"2024-09-05T13:14:52.543954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7626,"log_dups_size":0,"ondisk_log_size":7626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31211520,"num_objects":7620,"num_object_clones":0,"num_object_copies":15240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5636,"num_write_kb":22544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7713","reported_seq":7836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.445937+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:52.445937+0000","last_peered":"2024-09-05T13:14:52.445937+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:52.445937+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5524,"num_write_kb":22096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3217}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7663","reported_seq":7728,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:52.531423+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:52.531423+0000","last_peered":"2024-09-05T13:14:52.531423+0000","last_clean":"2024-09-05T13:14:52.531423+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:52.531423+0000","last_undegraded":"2024-09-05T13:14:52.531423+0000","last_fullsized":"2024-09-05T13:14:52.531423+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7663,"log_dups_size":0,"ondisk_log_size":7663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31387648,"num_objects":7663,"num_object_clones":0,"num_object_copies":15326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7663,"num_write_kb":30652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7779","reported_seq":7908,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:56.832476+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:56.832476+0000","last_peered":"2024-09-05T13:14:56.832476+0000","last_clean":"2024-09-05T13:14:56.832476+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:56.832476+0000","last_undegraded":"2024-09-05T13:14:56.832476+0000","last_fullsized":"2024-09-05T13:14:56.832476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7779,"log_dups_size":0,"ondisk_log_size":7779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31834112,"num_objects":7772,"num_object_clones":0,"num_object_copies":15544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7857","reported_seq":7990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.868719+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:56.868719+0000","last_peered":"2024-09-05T13:14:56.868719+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:56.868719+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7857,"log_dups_size":0,"ondisk_log_size":7857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32161792,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":17248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3401},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7713","reported_seq":7846,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.894003+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:56.894003+0000","last_peered":"2024-09-05T13:14:56.894003+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:56.894003+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31567872,"num_objects":7707,"num_object_clones":0,"num_object_copies":15414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3271},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7827","reported_seq":7953,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:56.906919+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:56.906919+0000","last_peered":"2024-09-05T13:14:56.906919+0000","last_clean":"2024-09-05T13:14:56.906919+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:56.906919+0000","last_undegraded":"2024-09-05T13:14:56.906919+0000","last_fullsized":"2024-09-05T13:14:56.906919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32051200,"num_objects":7825,"num_object_clones":0,"num_object_copies":15650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4437,"num_write_kb":17748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7607","reported_seq":7737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.451297+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:52.451297+0000","last_peered":"2024-09-05T13:14:52.451297+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:52.451297+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7749","reported_seq":7882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.821658+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:56.821658+0000","last_peered":"2024-09-05T13:14:56.821658+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:56.821658+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7749,"log_dups_size":0,"ondisk_log_size":7749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31707136,"num_objects":7741,"num_object_clones":0,"num_object_copies":15482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4116,"num_write_kb":16464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3287},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7631","reported_seq":12100,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.513725+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:52.513725+0000","last_peered":"2024-09-05T13:14:52.513725+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:52.513725+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7631,"log_dups_size":0,"ondisk_log_size":7631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31236096,"num_objects":7626,"num_object_clones":0,"num_object_copies":15252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":7626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5226,"num_write_kb":20904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2180,"num_bytes_recovered":8929280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2205},{"shards":"2","objects":5419}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7541","reported_seq":7659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.539945+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:52.539945+0000","last_peered":"2024-09-05T13:14:52.539945+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:52.539945+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5050,"num_write_kb":20200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3148}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7768","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.881293+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:56.881293+0000","last_peered":"2024-09-05T13:14:56.881293+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:56.881293+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7768,"log_dups_size":0,"ondisk_log_size":7768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31809536,"num_objects":7766,"num_object_clones":0,"num_object_copies":15532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3919,"num_write_kb":15676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3347},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7830","reported_seq":8012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.803103+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:56.803103+0000","last_peered":"2024-09-05T13:14:56.803103+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:56.803103+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32038912,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3729,"num_write_kb":14916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3378},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7867","reported_seq":8001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.752589+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:56.752589+0000","last_peered":"2024-09-05T13:14:56.752589+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:56.752589+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7867,"log_dups_size":0,"ondisk_log_size":7867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3997,"num_write_kb":15988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3297},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7905","reported_seq":8040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.863976+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:56.863976+0000","last_peered":"2024-09-05T13:14:56.863976+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:56.863976+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7905,"log_dups_size":0,"ondisk_log_size":7905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32330211,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4573,"num_write_kb":18292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3329},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506733027,"num_objects":123715,"num_object_clones":0,"num_object_copies":247430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":123715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79514,"num_write_kb":318056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":8941568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021276160,"data_stored":1021185835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123796,"ondisk_log_size":123796,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617536,"kb_used_data":387312,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93754304,"statfs":{"total":96636764160,"available":96004407296,"internally_reserved":0,"allocated":396607488,"data_stored":396362398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":235728557},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715560,"kb_used_data":445144,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93656280,"statfs":{"total":96636764160,"available":95904030720,"internally_reserved":0,"allocated":455827456,"data_stored":455578289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,2,1,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330324,"kb_used_data":163972,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94041516,"statfs":{"total":96636764160,"available":96298512384,"internally_reserved":0,"allocated":167907328,"data_stored":167578675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170323631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455569408,"data_stored":455565795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169017344,"data_stored":168934245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396689408,"data_stored":396685795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:58.754 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:14:59.230 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 146 20 125603 125583 3.35896 1.77734 0.0406342 0.0232526 2024-09-05T13:14:59.317 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:14:59.318 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:14:59.343 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-05T13:14:57.558249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506733046,"num_objects":123717,"num_object_clones":0,"num_object_copies":247434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":123717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79516,"num_write_kb":318058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2184,"num_bytes_recovered":8941587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123799,"ondisk_log_size":123799,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1663420,"kb_used_data":996428,"kb_used_omap":12,"kb_used_meta":666931,"kb_avail":281452100,"statfs":{"total":289910292480,"available":288206950400,"internally_reserved":0,"allocated":1020342272,"data_stored":1019519362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":682937350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,8,4,8,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26808320,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6545,"num_write_kb":26180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004265"},"pg_stats":[{"pgid":"2.f","version":"43'7720","reported_seq":7789,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.875703+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:56.875703+0000","last_peered":"2024-09-05T13:14:56.875703+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:56.875703+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7720,"log_dups_size":0,"ondisk_log_size":7720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31621120,"num_objects":7720,"num_object_clones":0,"num_object_copies":15440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7720,"num_write_kb":30880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3299},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7626","reported_seq":7725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:52.543954+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:52.543954+0000","last_peered":"2024-09-05T13:14:52.543954+0000","last_clean":"2024-09-05T13:14:52.543954+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:52.543954+0000","last_undegraded":"2024-09-05T13:14:52.543954+0000","last_fullsized":"2024-09-05T13:14:52.543954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7626,"log_dups_size":0,"ondisk_log_size":7626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31211520,"num_objects":7620,"num_object_clones":0,"num_object_copies":15240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5636,"num_write_kb":22544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7713","reported_seq":7836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.445937+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:52.445937+0000","last_peered":"2024-09-05T13:14:52.445937+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:52.445937+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5524,"num_write_kb":22096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3217}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7663","reported_seq":7728,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:52.531423+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:52.531423+0000","last_peered":"2024-09-05T13:14:52.531423+0000","last_clean":"2024-09-05T13:14:52.531423+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:52.531423+0000","last_undegraded":"2024-09-05T13:14:52.531423+0000","last_fullsized":"2024-09-05T13:14:52.531423+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7663,"log_dups_size":0,"ondisk_log_size":7663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31387648,"num_objects":7663,"num_object_clones":0,"num_object_copies":15326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7663,"num_write_kb":30652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7779","reported_seq":7908,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:56.832476+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:56.832476+0000","last_peered":"2024-09-05T13:14:56.832476+0000","last_clean":"2024-09-05T13:14:56.832476+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:56.832476+0000","last_undegraded":"2024-09-05T13:14:56.832476+0000","last_fullsized":"2024-09-05T13:14:56.832476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7779,"log_dups_size":0,"ondisk_log_size":7779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31834112,"num_objects":7772,"num_object_clones":0,"num_object_copies":15544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7857","reported_seq":7990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.868719+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:56.868719+0000","last_peered":"2024-09-05T13:14:56.868719+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:56.868719+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7857,"log_dups_size":0,"ondisk_log_size":7857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32161792,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":17248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3401},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7713","reported_seq":7846,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.894003+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:56.894003+0000","last_peered":"2024-09-05T13:14:56.894003+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:56.894003+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31567872,"num_objects":7707,"num_object_clones":0,"num_object_copies":15414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3271},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7827","reported_seq":7953,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:56.906919+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:56.906919+0000","last_peered":"2024-09-05T13:14:56.906919+0000","last_clean":"2024-09-05T13:14:56.906919+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:56.906919+0000","last_undegraded":"2024-09-05T13:14:56.906919+0000","last_fullsized":"2024-09-05T13:14:56.906919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32051200,"num_objects":7825,"num_object_clones":0,"num_object_copies":15650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4437,"num_write_kb":17748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7607","reported_seq":7737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.451297+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:52.451297+0000","last_peered":"2024-09-05T13:14:52.451297+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:52.451297+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5261,"num_write_kb":21044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7749","reported_seq":7882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.821658+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:56.821658+0000","last_peered":"2024-09-05T13:14:56.821658+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:56.821658+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7749,"log_dups_size":0,"ondisk_log_size":7749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31707136,"num_objects":7741,"num_object_clones":0,"num_object_copies":15482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4116,"num_write_kb":16464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3287},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7631","reported_seq":12100,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.513725+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:52.513725+0000","last_peered":"2024-09-05T13:14:52.513725+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:52.513725+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7631,"log_dups_size":0,"ondisk_log_size":7631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31236096,"num_objects":7626,"num_object_clones":0,"num_object_copies":15252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":7626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5226,"num_write_kb":20904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2180,"num_bytes_recovered":8929280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2205},{"shards":"2","objects":5419}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7541","reported_seq":7659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:52.539945+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:52.539945+0000","last_peered":"2024-09-05T13:14:52.539945+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:52.539945+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5050,"num_write_kb":20200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3148}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7768","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.881293+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:56.881293+0000","last_peered":"2024-09-05T13:14:56.881293+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:56.881293+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7768,"log_dups_size":0,"ondisk_log_size":7768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31809536,"num_objects":7766,"num_object_clones":0,"num_object_copies":15532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3919,"num_write_kb":15676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3347},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7830","reported_seq":8012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.803103+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:56.803103+0000","last_peered":"2024-09-05T13:14:56.803103+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:56.803103+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32038912,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3729,"num_write_kb":14916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3378},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7867","reported_seq":8001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.752589+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:56.752589+0000","last_peered":"2024-09-05T13:14:56.752589+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:56.752589+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7867,"log_dups_size":0,"ondisk_log_size":7867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3997,"num_write_kb":15988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3297},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7905","reported_seq":8040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.863976+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:56.863976+0000","last_peered":"2024-09-05T13:14:56.863976+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:56.863976+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7905,"log_dups_size":0,"ondisk_log_size":7905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32330211,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4573,"num_write_kb":18292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3329},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506733027,"num_objects":123715,"num_object_clones":0,"num_object_copies":247430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2205,"num_objects_unfound":0,"num_objects_dirty":123715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79514,"num_write_kb":318056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2183,"num_bytes_recovered":8941568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021276160,"data_stored":1021185835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123796,"ondisk_log_size":123796,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617536,"kb_used_data":387312,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93754304,"statfs":{"total":96636764160,"available":96004407296,"internally_reserved":0,"allocated":396607488,"data_stored":396362398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":235728557},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715560,"kb_used_data":445144,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93656280,"statfs":{"total":96636764160,"available":95904030720,"internally_reserved":0,"allocated":455827456,"data_stored":455578289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,2,1,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330324,"kb_used_data":163972,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94041516,"statfs":{"total":96636764160,"available":96298512384,"internally_reserved":0,"allocated":167907328,"data_stored":167578675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170323631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455569408,"data_stored":455565795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169017344,"data_stored":168934245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396689408,"data_stored":396685795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:14:59.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:14:59.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7720", 'reported_seq': 7789, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:56.875703+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:14:56.875703+0000', 'last_peered': '2024-09-05T13:14:56.875703+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:14:56.875703+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 7720, 'log_dups_size': 0, 'ondisk_log_size': 7720, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31621120, 'num_objects': 7720, 'num_object_clones': 0, 'num_object_copies': 15440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7720, 'num_write_kb': 30880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3299}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:59.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:14:59.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7713", 'reported_seq': 7836, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:52.445937+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:14:52.445937+0000', 'last_peered': '2024-09-05T13:14:52.445937+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:14:52.445937+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 7713, 'log_dups_size': 0, 'ondisk_log_size': 7713, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31576064, 'num_objects': 7709, 'num_object_clones': 0, 'num_object_copies': 15418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5524, 'num_write_kb': 22096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 3217}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:59.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:14:59.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7857", 'reported_seq': 7990, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:56.868719+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:14:56.868719+0000', 'last_peered': '2024-09-05T13:14:56.868719+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:14:56.868719+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 7857, 'log_dups_size': 0, 'ondisk_log_size': 7857, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32161792, 'num_objects': 7852, 'num_object_clones': 0, 'num_object_copies': 15704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7852, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4312, 'num_write_kb': 17248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3401}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:59.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7713", 'reported_seq': 7846, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:56.894003+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:14:56.894003+0000', 'last_peered': '2024-09-05T13:14:56.894003+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:14:56.894003+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 7713, 'log_dups_size': 0, 'ondisk_log_size': 7713, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31567872, 'num_objects': 7707, 'num_object_clones': 0, 'num_object_copies': 15414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7707, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4032, 'num_write_kb': 16128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3271}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7607", 'reported_seq': 7737, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:52.451297+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:14:52.451297+0000', 'last_peered': '2024-09-05T13:14:52.451297+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:14:52.451297+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 7607, 'log_dups_size': 0, 'ondisk_log_size': 7607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31129600, 'num_objects': 7600, 'num_object_clones': 0, 'num_object_copies': 15200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5261, 'num_write_kb': 21044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 3149}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7749", 'reported_seq': 7882, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:56.821658+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:14:56.821658+0000', 'last_peered': '2024-09-05T13:14:56.821658+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:14:56.821658+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 7749, 'log_dups_size': 0, 'ondisk_log_size': 7749, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31707136, 'num_objects': 7741, 'num_object_clones': 0, 'num_object_copies': 15482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4116, 'num_write_kb': 16464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3287}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7631", 'reported_seq': 12100, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:52.513725+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:14:52.513725+0000', 'last_peered': '2024-09-05T13:14:52.513725+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:14:52.513725+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 7631, 'log_dups_size': 0, 'ondisk_log_size': 7631, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31236096, 'num_objects': 7626, 'num_object_clones': 0, 'num_object_copies': 15252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2205, 'num_objects_unfound': 0, 'num_objects_dirty': 7626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5226, 'num_write_kb': 20904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2180, 'num_bytes_recovered': 8929280, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2205}, {'shards': '2', 'objects': 5419}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7541", 'reported_seq': 7659, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:52.539945+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:14:52.539945+0000', 'last_peered': '2024-09-05T13:14:52.539945+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:14:52.539945+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 7541, 'log_dups_size': 0, 'ondisk_log_size': 7541, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30863360, 'num_objects': 7535, 'num_object_clones': 0, 'num_object_copies': 15070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5050, 'num_write_kb': 20200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 3148}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:14:59.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7768", 'reported_seq': 7903, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:56.881293+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:14:56.881293+0000', 'last_peered': '2024-09-05T13:14:56.881293+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:14:56.881293+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 7768, 'log_dups_size': 0, 'ondisk_log_size': 7768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31809536, 'num_objects': 7766, 'num_object_clones': 0, 'num_object_copies': 15532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7766, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3919, 'num_write_kb': 15676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3347}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:59.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:14:59.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7830", 'reported_seq': 8012, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:56.803103+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:14:56.803103+0000', 'last_peered': '2024-09-05T13:14:56.803103+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:14:56.803103+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 7830, 'log_dups_size': 0, 'ondisk_log_size': 7830, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32038912, 'num_objects': 7822, 'num_object_clones': 0, 'num_object_copies': 15644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3729, 'num_write_kb': 14916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3378}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:59.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:14:59.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7867", 'reported_seq': 8001, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:56.752589+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:14:56.752589+0000', 'last_peered': '2024-09-05T13:14:56.752589+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:14:56.752589+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 7867, 'log_dups_size': 0, 'ondisk_log_size': 7867, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32206848, 'num_objects': 7863, 'num_object_clones': 0, 'num_object_copies': 15726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3997, 'num_write_kb': 15988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3297}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:59.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:14:59.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7905", 'reported_seq': 8040, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:14:56.863976+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:14:56.863976+0000', 'last_peered': '2024-09-05T13:14:56.863976+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:14:56.863976+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 7905, 'log_dups_size': 0, 'ondisk_log_size': 7905, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32330211, 'num_objects': 7894, 'num_object_clones': 0, 'num_object_copies': 15788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4573, 'num_write_kb': 18292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3329}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:14:59.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:00.022 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:00.022 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/249180 objects degraded (19.708%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":226,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":124590,"data_bytes":510308854,"bytes_used":1712545792,"bytes_avail":288197746688,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":249180,"degraded_ratio":0.19707841720844368,"misplaced_objects":2062,"misplaced_total":249180,"misplaced_ratio":0.0082751424672927194,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":90420,"recovering_keys_per_sec":0,"num_objects_recovered":267,"num_bytes_recovered":1085440,"num_keys_recovered":0,"write_bytes_sec":1666809,"read_op_per_sec":0,"write_op_per_sec":406},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:00.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:00.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:00.230 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 147 20 126046 126026 3.34788 1.73047 0.0521231 0.0233267 2024-09-05T13:15:00.580 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:00.611 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-05T13:14:59.559037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510308854,"num_objects":124590,"num_object_clones":0,"num_object_copies":249180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2062,"num_objects_unfound":0,"num_objects_dirty":124590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80389,"num_write_kb":321550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2329,"num_bytes_recovered":9527315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124672,"ondisk_log_size":124672,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1672408,"kb_used_data":1001256,"kb_used_omap":12,"kb_used_meta":671091,"kb_avail":281443112,"statfs":{"total":289910292480,"available":288197746688,"internally_reserved":0,"allocated":1025286144,"data_stored":1024463234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":687197190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,10,9,8,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-265,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":1085440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004344"},"pg_stats":[{"pgid":"2.f","version":"43'7720","reported_seq":7789,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.875703+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:14:56.875703+0000","last_peered":"2024-09-05T13:14:56.875703+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:14:56.875703+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7720,"log_dups_size":0,"ondisk_log_size":7720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31621120,"num_objects":7720,"num_object_clones":0,"num_object_copies":15440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7720,"num_write_kb":30880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3299},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7774","reported_seq":7874,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:57.516325+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:14:57.516325+0000","last_peered":"2024-09-05T13:14:57.516325+0000","last_clean":"2024-09-05T13:14:57.516325+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:14:57.516325+0000","last_undegraded":"2024-09-05T13:14:57.516325+0000","last_fullsized":"2024-09-05T13:14:57.516325+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7774,"log_dups_size":0,"ondisk_log_size":7774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31817728,"num_objects":7768,"num_object_clones":0,"num_object_copies":15536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":23136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7869","reported_seq":7990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:57.565724+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:14:57.565724+0000","last_peered":"2024-09-05T13:14:57.565724+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:14:57.565724+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5680,"num_write_kb":22720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3373}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7801","reported_seq":7866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:57.533051+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:14:57.533051+0000","last_peered":"2024-09-05T13:14:57.533051+0000","last_clean":"2024-09-05T13:14:57.533051+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:14:57.533051+0000","last_undegraded":"2024-09-05T13:14:57.533051+0000","last_fullsized":"2024-09-05T13:14:57.533051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7801,"log_dups_size":0,"ondisk_log_size":7801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31952896,"num_objects":7801,"num_object_clones":0,"num_object_copies":15602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7801,"num_write_kb":31204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7779","reported_seq":7908,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:56.832476+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:14:56.832476+0000","last_peered":"2024-09-05T13:14:56.832476+0000","last_clean":"2024-09-05T13:14:56.832476+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:14:56.832476+0000","last_undegraded":"2024-09-05T13:14:56.832476+0000","last_fullsized":"2024-09-05T13:14:56.832476+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7779,"log_dups_size":0,"ondisk_log_size":7779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31834112,"num_objects":7772,"num_object_clones":0,"num_object_copies":15544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7857","reported_seq":7990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.868719+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:14:56.868719+0000","last_peered":"2024-09-05T13:14:56.868719+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:14:56.868719+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7857,"log_dups_size":0,"ondisk_log_size":7857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32161792,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":17248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3401},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7713","reported_seq":7846,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.894003+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:14:56.894003+0000","last_peered":"2024-09-05T13:14:56.894003+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:14:56.894003+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31567872,"num_objects":7707,"num_object_clones":0,"num_object_copies":15414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3271},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7827","reported_seq":7953,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:14:56.906919+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:14:56.906919+0000","last_peered":"2024-09-05T13:14:56.906919+0000","last_clean":"2024-09-05T13:14:56.906919+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:14:56.906919+0000","last_undegraded":"2024-09-05T13:14:56.906919+0000","last_fullsized":"2024-09-05T13:14:56.906919+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32051200,"num_objects":7825,"num_object_clones":0,"num_object_copies":15650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4437,"num_write_kb":17748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7770","reported_seq":7900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:57.533522+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:14:57.533522+0000","last_peered":"2024-09-05T13:14:57.533522+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:14:57.533522+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7770,"log_dups_size":0,"ondisk_log_size":7770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31797248,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7749","reported_seq":7882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.821658+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:14:56.821658+0000","last_peered":"2024-09-05T13:14:56.821658+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:14:56.821658+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7749,"log_dups_size":0,"ondisk_log_size":7749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31707136,"num_objects":7741,"num_object_clones":0,"num_object_copies":15482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4116,"num_write_kb":16464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3287},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7764","reported_seq":12526,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:57.558569+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:14:57.558569+0000","last_peered":"2024-09-05T13:14:57.558569+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:14:57.558569+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7764,"log_dups_size":0,"ondisk_log_size":7764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31780864,"num_objects":7759,"num_object_clones":0,"num_object_copies":15518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2062,"num_objects_unfound":0,"num_objects_dirty":7759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":21436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2325,"num_bytes_recovered":9515008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2062},{"shards":"2","objects":5695}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7676","reported_seq":7794,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:57.444955+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:14:57.444955+0000","last_peered":"2024-09-05T13:14:57.444955+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:14:57.444955+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7676,"log_dups_size":0,"ondisk_log_size":7676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31416320,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5185,"num_write_kb":20740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3283}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7768","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.881293+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:14:56.881293+0000","last_peered":"2024-09-05T13:14:56.881293+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:14:56.881293+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7768,"log_dups_size":0,"ondisk_log_size":7768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31809536,"num_objects":7766,"num_object_clones":0,"num_object_copies":15532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3919,"num_write_kb":15676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3347},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7830","reported_seq":8012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.803103+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:14:56.803103+0000","last_peered":"2024-09-05T13:14:56.803103+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:14:56.803103+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32038912,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3729,"num_write_kb":14916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3378},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7867","reported_seq":8001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.752589+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:14:56.752589+0000","last_peered":"2024-09-05T13:14:56.752589+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:14:56.752589+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":7867,"log_dups_size":0,"ondisk_log_size":7867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3997,"num_write_kb":15988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3297},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7905","reported_seq":8040,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:14:56.863976+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:14:56.863976+0000","last_peered":"2024-09-05T13:14:56.863976+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:14:56.863976+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":7905,"log_dups_size":0,"ondisk_log_size":7905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32330211,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4573,"num_write_kb":18292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3329},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":510308835,"num_objects":124588,"num_object_clones":0,"num_object_copies":249176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":2062,"num_objects_unfound":0,"num_objects_dirty":124588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80387,"num_write_kb":321548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2328,"num_bytes_recovered":9527296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1026154496,"data_stored":1026064171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124669,"ondisk_log_size":124669,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626524,"kb_used_data":392140,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93745316,"statfs":{"total":96636764160,"available":95995203584,"internally_reserved":0,"allocated":401551360,"data_stored":401306270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":239988397},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715560,"kb_used_data":445144,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93656280,"statfs":{"total":96636764160,"available":95904030720,"internally_reserved":0,"allocated":455827456,"data_stored":455578289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,2,1,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330324,"kb_used_data":163972,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94041516,"statfs":{"total":96636764160,"available":96298512384,"internally_reserved":0,"allocated":167907328,"data_stored":167578675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170323631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455569408,"data_stored":455565795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":169017344,"data_stored":168934245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401567744,"data_stored":401564131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:01.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 148 20 126531 126511 3.33804 1.89453 0.0605184 0.0233971 2024-09-05T13:15:02.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 149 20 127041 127021 3.32901 1.99219 0.0332096 0.0234596 2024-09-05T13:15:03.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 150 20 127538 127518 3.31976 1.94141 0.109551 0.0235225 2024-09-05T13:15:03.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:04.181 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:04.203 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-05T13:15:03.560394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520176118,"num_objects":126999,"num_object_clones":0,"num_object_copies":253998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":126999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82798,"num_write_kb":331186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2460,"num_bytes_recovered":10063891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127081,"ondisk_log_size":127081,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1710456,"kb_used_data":1021256,"kb_used_omap":12,"kb_used_meta":689139,"kb_avail":281405064,"statfs":{"total":289910292480,"available":288158785536,"internally_reserved":0,"allocated":1045766144,"data_stored":1044950460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":705678342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,12,7,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-396,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7294,"num_write_kb":29176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":1622016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004447"},"pg_stats":[{"pgid":"2.f","version":"43'7866","reported_seq":7936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.709989+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:01.709989+0000","last_peered":"2024-09-05T13:15:01.709989+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:01.709989+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7866,"log_dups_size":0,"ondisk_log_size":7866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7917","reported_seq":8017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:02.487990+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:02.487990+0000","last_peered":"2024-09-05T13:15:02.487990+0000","last_clean":"2024-09-05T13:15:02.487990+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:02.487990+0000","last_undegraded":"2024-09-05T13:15:02.487990+0000","last_fullsized":"2024-09-05T13:15:02.487990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8035","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.520570+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:02.520570+0000","last_peered":"2024-09-05T13:15:02.520570+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:02.520570+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8035,"log_dups_size":0,"ondisk_log_size":8035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32894976,"num_objects":8031,"num_object_clones":0,"num_object_copies":16062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5846,"num_write_kb":23384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7949","reported_seq":8014,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:02.537694+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:02.537694+0000","last_peered":"2024-09-05T13:15:02.537694+0000","last_clean":"2024-09-05T13:15:02.537694+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:02.537694+0000","last_undegraded":"2024-09-05T13:15:02.537694+0000","last_fullsized":"2024-09-05T13:15:02.537694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32559104,"num_objects":7949,"num_object_clones":0,"num_object_copies":15898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7931","reported_seq":8060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:01.902926+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:01.902926+0000","last_peered":"2024-09-05T13:15:01.902926+0000","last_clean":"2024-09-05T13:15:01.902926+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:01.902926+0000","last_undegraded":"2024-09-05T13:15:01.902926+0000","last_fullsized":"2024-09-05T13:15:01.902926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7931,"log_dups_size":0,"ondisk_log_size":7931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32456704,"num_objects":7924,"num_object_clones":0,"num_object_copies":15848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":17884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7997","reported_seq":8130,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.794819+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:01.794819+0000","last_peered":"2024-09-05T13:15:01.794819+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:01.794819+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7997,"log_dups_size":0,"ondisk_log_size":7997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32735232,"num_objects":7992,"num_object_clones":0,"num_object_copies":15984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4452,"num_write_kb":17808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3541},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7855","reported_seq":7990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.871308+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:01.871308+0000","last_peered":"2024-09-05T13:15:01.871308+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:01.871308+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4174,"num_write_kb":16696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3413},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8001","reported_seq":8127,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:01.894574+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:01.894574+0000","last_peered":"2024-09-05T13:15:01.894574+0000","last_clean":"2024-09-05T13:15:01.894574+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:01.894574+0000","last_undegraded":"2024-09-05T13:15:01.894574+0000","last_fullsized":"2024-09-05T13:15:01.894574+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8001,"log_dups_size":0,"ondisk_log_size":8001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32763904,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7900","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.472391+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:02.472391+0000","last_peered":"2024-09-05T13:15:02.472391+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:02.472391+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7900,"log_dups_size":0,"ondisk_log_size":7900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32329728,"num_objects":7893,"num_object_clones":0,"num_object_copies":15786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5554,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7897","reported_seq":8029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.844723+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:01.844723+0000","last_peered":"2024-09-05T13:15:01.844723+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:01.844723+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4264,"num_write_kb":17056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3435},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7928","reported_seq":12952,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.561804+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:02.561804+0000","last_peered":"2024-09-05T13:15:02.561804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:02.561804+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32452608,"num_objects":7923,"num_object_clones":0,"num_object_copies":15846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":7923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5523,"num_write_kb":22092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2456,"num_bytes_recovered":10051584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1931},{"shards":"2","objects":5990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7829","reported_seq":7947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.559811+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:02.559811+0000","last_peered":"2024-09-05T13:15:02.559811+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:02.559811+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7829,"log_dups_size":0,"ondisk_log_size":7829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5338,"num_write_kb":21352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3436}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7916","reported_seq":8049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.846133+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:01.846133+0000","last_peered":"2024-09-05T13:15:01.846133+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:01.846133+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7916,"log_dups_size":0,"ondisk_log_size":7916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32415744,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4067,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3495},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7984","reported_seq":8166,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.872768+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:01.872768+0000","last_peered":"2024-09-05T13:15:01.872768+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:01.872768+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7984,"log_dups_size":0,"ondisk_log_size":7984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3883,"num_write_kb":15532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8035","reported_seq":8169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.867994+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:01.867994+0000","last_peered":"2024-09-05T13:15:01.867994+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:01.867994+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8035,"log_dups_size":0,"ondisk_log_size":8035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32894976,"num_objects":8031,"num_object_clones":0,"num_object_copies":16062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4165,"num_write_kb":16660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3465},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8038","reported_seq":8173,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.850732+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:01.850732+0000","last_peered":"2024-09-05T13:15:01.850732+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:01.850732+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8038,"log_dups_size":0,"ondisk_log_size":8038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874979,"num_objects":8027,"num_object_clones":0,"num_object_copies":16054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4706,"num_write_kb":18824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3462},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":520176099,"num_objects":126997,"num_object_clones":0,"num_object_copies":253994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":126997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82796,"num_write_kb":331184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2459,"num_bytes_recovered":10063872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1046589440,"data_stored":1046506341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127078,"ondisk_log_size":127078,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635576,"kb_used_data":397032,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93736264,"statfs":{"total":96636764160,"available":95985934336,"internally_reserved":0,"allocated":406560768,"data_stored":406315678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744960,"kb_used_data":452336,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93626880,"statfs":{"total":96636764160,"available":95873925120,"internally_reserved":0,"allocated":463192064,"data_stored":462942897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329920,"kb_used_data":171888,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94041920,"statfs":{"total":96636764160,"available":96298926080,"internally_reserved":0,"allocated":176013312,"data_stored":175691885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161803951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,1,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462880768,"data_stored":462877155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":177123328,"data_stored":177047455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406585344,"data_stored":406581731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:04.204 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:04.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 151 20 128008 127988 3.30994 1.83594 0.0432792 0.023594 2024-09-05T13:15:04.773 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:04.773 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:04.807 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-05T13:15:03.560394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520176118,"num_objects":126999,"num_object_clones":0,"num_object_copies":253998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":126999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82798,"num_write_kb":331186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2460,"num_bytes_recovered":10063891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127081,"ondisk_log_size":127081,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1710456,"kb_used_data":1021256,"kb_used_omap":12,"kb_used_meta":689139,"kb_avail":281405064,"statfs":{"total":289910292480,"available":288158785536,"internally_reserved":0,"allocated":1045766144,"data_stored":1044950460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":705678342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,12,7,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-396,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7294,"num_write_kb":29176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":1622016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004447"},"pg_stats":[{"pgid":"2.f","version":"43'7866","reported_seq":7936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.709989+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:01.709989+0000","last_peered":"2024-09-05T13:15:01.709989+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:01.709989+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7866,"log_dups_size":0,"ondisk_log_size":7866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7917","reported_seq":8017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:02.487990+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:02.487990+0000","last_peered":"2024-09-05T13:15:02.487990+0000","last_clean":"2024-09-05T13:15:02.487990+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:02.487990+0000","last_undegraded":"2024-09-05T13:15:02.487990+0000","last_fullsized":"2024-09-05T13:15:02.487990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8035","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.520570+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:02.520570+0000","last_peered":"2024-09-05T13:15:02.520570+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:02.520570+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8035,"log_dups_size":0,"ondisk_log_size":8035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32894976,"num_objects":8031,"num_object_clones":0,"num_object_copies":16062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5846,"num_write_kb":23384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7949","reported_seq":8014,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:02.537694+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:02.537694+0000","last_peered":"2024-09-05T13:15:02.537694+0000","last_clean":"2024-09-05T13:15:02.537694+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:02.537694+0000","last_undegraded":"2024-09-05T13:15:02.537694+0000","last_fullsized":"2024-09-05T13:15:02.537694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32559104,"num_objects":7949,"num_object_clones":0,"num_object_copies":15898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7931","reported_seq":8060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:01.902926+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:01.902926+0000","last_peered":"2024-09-05T13:15:01.902926+0000","last_clean":"2024-09-05T13:15:01.902926+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:01.902926+0000","last_undegraded":"2024-09-05T13:15:01.902926+0000","last_fullsized":"2024-09-05T13:15:01.902926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7931,"log_dups_size":0,"ondisk_log_size":7931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32456704,"num_objects":7924,"num_object_clones":0,"num_object_copies":15848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":17884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7997","reported_seq":8130,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.794819+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:01.794819+0000","last_peered":"2024-09-05T13:15:01.794819+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:01.794819+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7997,"log_dups_size":0,"ondisk_log_size":7997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32735232,"num_objects":7992,"num_object_clones":0,"num_object_copies":15984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4452,"num_write_kb":17808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3541},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7855","reported_seq":7990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.871308+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:01.871308+0000","last_peered":"2024-09-05T13:15:01.871308+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:01.871308+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4174,"num_write_kb":16696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3413},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8001","reported_seq":8127,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:01.894574+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:01.894574+0000","last_peered":"2024-09-05T13:15:01.894574+0000","last_clean":"2024-09-05T13:15:01.894574+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:01.894574+0000","last_undegraded":"2024-09-05T13:15:01.894574+0000","last_fullsized":"2024-09-05T13:15:01.894574+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8001,"log_dups_size":0,"ondisk_log_size":8001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32763904,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7900","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.472391+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:02.472391+0000","last_peered":"2024-09-05T13:15:02.472391+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:02.472391+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7900,"log_dups_size":0,"ondisk_log_size":7900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32329728,"num_objects":7893,"num_object_clones":0,"num_object_copies":15786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5554,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7897","reported_seq":8029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.844723+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:01.844723+0000","last_peered":"2024-09-05T13:15:01.844723+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:01.844723+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4264,"num_write_kb":17056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3435},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7928","reported_seq":12952,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.561804+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:02.561804+0000","last_peered":"2024-09-05T13:15:02.561804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:02.561804+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32452608,"num_objects":7923,"num_object_clones":0,"num_object_copies":15846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":7923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5523,"num_write_kb":22092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2456,"num_bytes_recovered":10051584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1931},{"shards":"2","objects":5990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7829","reported_seq":7947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.559811+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:02.559811+0000","last_peered":"2024-09-05T13:15:02.559811+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:02.559811+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7829,"log_dups_size":0,"ondisk_log_size":7829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5338,"num_write_kb":21352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3436}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7916","reported_seq":8049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.846133+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:01.846133+0000","last_peered":"2024-09-05T13:15:01.846133+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:01.846133+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7916,"log_dups_size":0,"ondisk_log_size":7916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32415744,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4067,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3495},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7984","reported_seq":8166,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.872768+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:01.872768+0000","last_peered":"2024-09-05T13:15:01.872768+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:01.872768+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7984,"log_dups_size":0,"ondisk_log_size":7984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3883,"num_write_kb":15532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8035","reported_seq":8169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.867994+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:01.867994+0000","last_peered":"2024-09-05T13:15:01.867994+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:01.867994+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8035,"log_dups_size":0,"ondisk_log_size":8035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32894976,"num_objects":8031,"num_object_clones":0,"num_object_copies":16062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4165,"num_write_kb":16660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3465},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8038","reported_seq":8173,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.850732+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:01.850732+0000","last_peered":"2024-09-05T13:15:01.850732+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:01.850732+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8038,"log_dups_size":0,"ondisk_log_size":8038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874979,"num_objects":8027,"num_object_clones":0,"num_object_copies":16054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4706,"num_write_kb":18824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3462},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":520176099,"num_objects":126997,"num_object_clones":0,"num_object_copies":253994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":126997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82796,"num_write_kb":331184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2459,"num_bytes_recovered":10063872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1046589440,"data_stored":1046506341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127078,"ondisk_log_size":127078,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635576,"kb_used_data":397032,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93736264,"statfs":{"total":96636764160,"available":95985934336,"internally_reserved":0,"allocated":406560768,"data_stored":406315678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744960,"kb_used_data":452336,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93626880,"statfs":{"total":96636764160,"available":95873925120,"internally_reserved":0,"allocated":463192064,"data_stored":462942897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329920,"kb_used_data":171888,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94041920,"statfs":{"total":96636764160,"available":96298926080,"internally_reserved":0,"allocated":176013312,"data_stored":175691885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161803951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,1,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462880768,"data_stored":462877155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":177123328,"data_stored":177047455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406585344,"data_stored":406581731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:04.808 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:04.808 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7866", 'reported_seq': 7936, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:01.709989+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:01.709989+0000', 'last_peered': '2024-09-05T13:15:01.709989+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:01.709989+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 7866, 'log_dups_size': 0, 'ondisk_log_size': 7866, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32219136, 'num_objects': 7866, 'num_object_clones': 0, 'num_object_copies': 15732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7866, 'num_write_kb': 31464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3445}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:04.808 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:04.808 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8035", 'reported_seq': 8157, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:02.520570+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:02.520570+0000', 'last_peered': '2024-09-05T13:15:02.520570+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:02.520570+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 8035, 'log_dups_size': 0, 'ondisk_log_size': 8035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32894976, 'num_objects': 8031, 'num_object_clones': 0, 'num_object_copies': 16062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8031, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5846, 'num_write_kb': 23384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 3539}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7997", 'reported_seq': 8130, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:01.794819+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:01.794819+0000', 'last_peered': '2024-09-05T13:15:01.794819+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:01.794819+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 7997, 'log_dups_size': 0, 'ondisk_log_size': 7997, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32735232, 'num_objects': 7992, 'num_object_clones': 0, 'num_object_copies': 15984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7992, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4452, 'num_write_kb': 17808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3541}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7855", 'reported_seq': 7990, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:01.871308+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:01.871308+0000', 'last_peered': '2024-09-05T13:15:01.871308+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:01.871308+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 7855, 'log_dups_size': 0, 'ondisk_log_size': 7855, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32149504, 'num_objects': 7849, 'num_object_clones': 0, 'num_object_copies': 15698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7849, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4174, 'num_write_kb': 16696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3413}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7900", 'reported_seq': 8030, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:02.472391+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:02.472391+0000', 'last_peered': '2024-09-05T13:15:02.472391+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:02.472391+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 7900, 'log_dups_size': 0, 'ondisk_log_size': 7900, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32329728, 'num_objects': 7893, 'num_object_clones': 0, 'num_object_copies': 15786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5554, 'num_write_kb': 22216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 3442}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7897", 'reported_seq': 8029, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:01.844723+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:01.844723+0000', 'last_peered': '2024-09-05T13:15:01.844723+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:01.844723+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 7897, 'log_dups_size': 0, 'ondisk_log_size': 7897, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32313344, 'num_objects': 7889, 'num_object_clones': 0, 'num_object_copies': 15778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4264, 'num_write_kb': 17056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3435}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:04.809 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7928", 'reported_seq': 12952, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:02.561804+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:02.561804+0000', 'last_peered': '2024-09-05T13:15:02.561804+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:02.561804+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 7928, 'log_dups_size': 0, 'ondisk_log_size': 7928, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32452608, 'num_objects': 7923, 'num_object_clones': 0, 'num_object_copies': 15846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1931, 'num_objects_unfound': 0, 'num_objects_dirty': 7923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5523, 'num_write_kb': 22092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2456, 'num_bytes_recovered': 10051584, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1931}, {'shards': '2', 'objects': 5990}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7829", 'reported_seq': 7947, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:02.559811+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:02.559811+0000', 'last_peered': '2024-09-05T13:15:02.559811+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:02.559811+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 7829, 'log_dups_size': 0, 'ondisk_log_size': 7829, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32043008, 'num_objects': 7823, 'num_object_clones': 0, 'num_object_copies': 15646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5338, 'num_write_kb': 21352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 3436}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7916", 'reported_seq': 8049, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:01.846133+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:01.846133+0000', 'last_peered': '2024-09-05T13:15:01.846133+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:01.846133+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 7916, 'log_dups_size': 0, 'ondisk_log_size': 7916, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32415744, 'num_objects': 7914, 'num_object_clones': 0, 'num_object_copies': 15828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7914, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4067, 'num_write_kb': 16268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3495}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7984", 'reported_seq': 8166, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:01.872768+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:01.872768+0000', 'last_peered': '2024-09-05T13:15:01.872768+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:01.872768+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 7984, 'log_dups_size': 0, 'ondisk_log_size': 7984, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32669696, 'num_objects': 7976, 'num_object_clones': 0, 'num_object_copies': 15952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3883, 'num_write_kb': 15532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3532}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8035", 'reported_seq': 8169, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:01.867994+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:01.867994+0000', 'last_peered': '2024-09-05T13:15:01.867994+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:01.867994+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 8035, 'log_dups_size': 0, 'ondisk_log_size': 8035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32894976, 'num_objects': 8031, 'num_object_clones': 0, 'num_object_copies': 16062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8031, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4165, 'num_write_kb': 16660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3465}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:04.810 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:04.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8038", 'reported_seq': 8173, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:01.850732+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:01.850732+0000', 'last_peered': '2024-09-05T13:15:01.850732+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:01.850732+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 8038, 'log_dups_size': 0, 'ondisk_log_size': 8038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32874979, 'num_objects': 8027, 'num_object_clones': 0, 'num_object_copies': 16054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8027, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4706, 'num_write_kb': 18824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3462}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:04.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:05.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 152 20 128452 128432 3.29957 1.73438 0.0518079 0.0236702 2024-09-05T13:15:05.536 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:05.536 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/253998 objects degraded (19.334%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":232,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":126999,"data_bytes":520176118,"bytes_used":1751506944,"bytes_avail":288158785536,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":253998,"degraded_ratio":0.19334010504019716,"misplaced_objects":1931,"misplaced_total":253998,"misplaced_ratio":0.0076024220663154828,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":135117,"recovering_keys_per_sec":0,"num_objects_recovered":398,"num_bytes_recovered":1622016,"num_keys_recovered":0,"write_bytes_sec":2488762,"read_op_per_sec":0,"write_op_per_sec":607},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:05.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:05.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:06.118 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:06.119 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:06.142 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-05T13:15:05.561254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520176118,"num_objects":126999,"num_object_clones":0,"num_object_copies":253998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":126999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82798,"num_write_kb":331186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2460,"num_bytes_recovered":10063891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127081,"ondisk_log_size":127081,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1723820,"kb_used_data":1030460,"kb_used_omap":12,"kb_used_meta":693299,"kb_avail":281391700,"statfs":{"total":289910292480,"available":288145100800,"internally_reserved":0,"allocated":1055191040,"data_stored":1054375356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":709938182},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,8,16,9,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19800064,"num_objects":4834,"num_object_clones":0,"num_object_copies":9668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4834,"num_write_kb":19336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004695"},"pg_stats":[{"pgid":"2.f","version":"43'7866","reported_seq":7936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.709989+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:01.709989+0000","last_peered":"2024-09-05T13:15:01.709989+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:01.709989+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7866,"log_dups_size":0,"ondisk_log_size":7866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32219136,"num_objects":7866,"num_object_clones":0,"num_object_copies":15732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3445},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7917","reported_seq":8017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:02.487990+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:02.487990+0000","last_peered":"2024-09-05T13:15:02.487990+0000","last_clean":"2024-09-05T13:15:02.487990+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:02.487990+0000","last_undegraded":"2024-09-05T13:15:02.487990+0000","last_fullsized":"2024-09-05T13:15:02.487990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32403456,"num_objects":7911,"num_object_clones":0,"num_object_copies":15822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8035","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.520570+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:02.520570+0000","last_peered":"2024-09-05T13:15:02.520570+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:02.520570+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8035,"log_dups_size":0,"ondisk_log_size":8035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32894976,"num_objects":8031,"num_object_clones":0,"num_object_copies":16062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5846,"num_write_kb":23384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3539}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7949","reported_seq":8014,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:02.537694+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:02.537694+0000","last_peered":"2024-09-05T13:15:02.537694+0000","last_clean":"2024-09-05T13:15:02.537694+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:02.537694+0000","last_undegraded":"2024-09-05T13:15:02.537694+0000","last_fullsized":"2024-09-05T13:15:02.537694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32559104,"num_objects":7949,"num_object_clones":0,"num_object_copies":15898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7931","reported_seq":8060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:01.902926+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:01.902926+0000","last_peered":"2024-09-05T13:15:01.902926+0000","last_clean":"2024-09-05T13:15:01.902926+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:01.902926+0000","last_undegraded":"2024-09-05T13:15:01.902926+0000","last_fullsized":"2024-09-05T13:15:01.902926+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":7931,"log_dups_size":0,"ondisk_log_size":7931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32456704,"num_objects":7924,"num_object_clones":0,"num_object_copies":15848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":17884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7997","reported_seq":8130,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.794819+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:01.794819+0000","last_peered":"2024-09-05T13:15:01.794819+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:01.794819+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":7997,"log_dups_size":0,"ondisk_log_size":7997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32735232,"num_objects":7992,"num_object_clones":0,"num_object_copies":15984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4452,"num_write_kb":17808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3541},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7855","reported_seq":7990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.871308+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:01.871308+0000","last_peered":"2024-09-05T13:15:01.871308+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:01.871308+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4174,"num_write_kb":16696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3413},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8001","reported_seq":8127,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:01.894574+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:01.894574+0000","last_peered":"2024-09-05T13:15:01.894574+0000","last_clean":"2024-09-05T13:15:01.894574+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:01.894574+0000","last_undegraded":"2024-09-05T13:15:01.894574+0000","last_fullsized":"2024-09-05T13:15:01.894574+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8001,"log_dups_size":0,"ondisk_log_size":8001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32763904,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7900","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.472391+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:02.472391+0000","last_peered":"2024-09-05T13:15:02.472391+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:02.472391+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":7900,"log_dups_size":0,"ondisk_log_size":7900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32329728,"num_objects":7893,"num_object_clones":0,"num_object_copies":15786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5554,"num_write_kb":22216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7897","reported_seq":8029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.844723+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:01.844723+0000","last_peered":"2024-09-05T13:15:01.844723+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:01.844723+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4264,"num_write_kb":17056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3435},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7928","reported_seq":12952,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.561804+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:02.561804+0000","last_peered":"2024-09-05T13:15:02.561804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:02.561804+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32452608,"num_objects":7923,"num_object_clones":0,"num_object_copies":15846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":7923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5523,"num_write_kb":22092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2456,"num_bytes_recovered":10051584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1931},{"shards":"2","objects":5990}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7829","reported_seq":7947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:02.559811+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:02.559811+0000","last_peered":"2024-09-05T13:15:02.559811+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:02.559811+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7829,"log_dups_size":0,"ondisk_log_size":7829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5338,"num_write_kb":21352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3436}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7916","reported_seq":8049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.846133+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:01.846133+0000","last_peered":"2024-09-05T13:15:01.846133+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:01.846133+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":7916,"log_dups_size":0,"ondisk_log_size":7916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32415744,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4067,"num_write_kb":16268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3495},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7984","reported_seq":8166,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.872768+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:01.872768+0000","last_peered":"2024-09-05T13:15:01.872768+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:01.872768+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":7984,"log_dups_size":0,"ondisk_log_size":7984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32669696,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3883,"num_write_kb":15532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8035","reported_seq":8169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.867994+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:01.867994+0000","last_peered":"2024-09-05T13:15:01.867994+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:01.867994+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8035,"log_dups_size":0,"ondisk_log_size":8035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32894976,"num_objects":8031,"num_object_clones":0,"num_object_copies":16062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4165,"num_write_kb":16660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3465},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8038","reported_seq":8173,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:01.850732+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:01.850732+0000","last_peered":"2024-09-05T13:15:01.850732+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:01.850732+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8038,"log_dups_size":0,"ondisk_log_size":8038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874979,"num_objects":8027,"num_object_clones":0,"num_object_copies":16054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4706,"num_write_kb":18824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3462},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":520176099,"num_objects":126997,"num_object_clones":0,"num_object_copies":253994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1931,"num_objects_unfound":0,"num_objects_dirty":126997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":82796,"num_write_kb":331184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2459,"num_bytes_recovered":10063872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1054621696,"data_stored":1054538597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127078,"ondisk_log_size":127078,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635576,"kb_used_data":397032,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93736264,"statfs":{"total":96636764160,"available":95985934336,"internally_reserved":0,"allocated":406560768,"data_stored":406315678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":244248237},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744960,"kb_used_data":452336,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93626880,"statfs":{"total":96636764160,"available":95873925120,"internally_reserved":0,"allocated":463192064,"data_stored":462942897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343284,"kb_used_data":181092,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94028556,"statfs":{"total":96636764160,"available":96285241344,"internally_reserved":0,"allocated":185438208,"data_stored":185116781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":166063791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,8,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462880768,"data_stored":462877155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185155584,"data_stored":185079711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406585344,"data_stored":406581731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:06.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 153 20 128914 128894 3.2898 1.80469 0.0198818 0.0237414 2024-09-05T13:15:07.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 154 20 129412 129392 3.28107 1.94531 0.02416 0.0238035 2024-09-05T13:15:08.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 155 20 129916 129896 3.27261 1.96875 0.0343185 0.0238657 2024-09-05T13:15:09.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:09.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 156 20 130409 130389 3.26398 1.92578 0.0748272 0.0239269 2024-09-05T13:15:09.680 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:09.683 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:09.704 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-05T13:15:09.562665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":529953270,"num_objects":129386,"num_object_clones":0,"num_object_copies":258772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":129386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85185,"num_write_kb":340734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2598,"num_bytes_recovered":10629139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129468,"ondisk_log_size":129468,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1745244,"kb_used_data":1042156,"kb_used_omap":12,"kb_used_meta":703027,"kb_avail":281370276,"statfs":{"total":289910292480,"available":288123162624,"internally_reserved":0,"allocated":1067167744,"data_stored":1066352060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":719899654},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,14,10,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23220224,"num_objects":5669,"num_object_clones":0,"num_object_copies":11338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-412,"num_objects_unfound":0,"num_objects_dirty":5669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5669,"num_write_kb":22676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1687552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004416"},"pg_stats":[{"pgid":"2.f","version":"43'8014","reported_seq":8084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.929435+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:06.929435+0000","last_peered":"2024-09-05T13:15:06.929435+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:06.929435+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8014,"num_write_kb":32056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3593},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8049","reported_seq":8149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:07.496008+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:07.496008+0000","last_peered":"2024-09-05T13:15:07.496008+0000","last_clean":"2024-09-05T13:15:07.496008+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:07.496008+0000","last_undegraded":"2024-09-05T13:15:07.496008+0000","last_fullsized":"2024-09-05T13:15:07.496008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6059,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8181","reported_seq":8304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.624380+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:07.624380+0000","last_peered":"2024-09-05T13:15:07.624380+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:07.624380+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8181,"log_dups_size":0,"ondisk_log_size":8181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33492992,"num_objects":8177,"num_object_clones":0,"num_object_copies":16354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5992,"num_write_kb":23968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3685}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8098","reported_seq":8161,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:07.630064+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:07.630064+0000","last_peered":"2024-09-05T13:15:07.630064+0000","last_clean":"2024-09-05T13:15:07.630064+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:07.630064+0000","last_undegraded":"2024-09-05T13:15:07.630064+0000","last_fullsized":"2024-09-05T13:15:07.630064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8098,"log_dups_size":0,"ondisk_log_size":8098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33169408,"num_objects":8098,"num_object_clones":0,"num_object_copies":16196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8098,"num_write_kb":32392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8081","reported_seq":8210,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:06.887480+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:06.887480+0000","last_peered":"2024-09-05T13:15:06.887480+0000","last_clean":"2024-09-05T13:15:06.887480+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:06.887480+0000","last_undegraded":"2024-09-05T13:15:06.887480+0000","last_fullsized":"2024-09-05T13:15:06.887480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8081,"log_dups_size":0,"ondisk_log_size":8081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33071104,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4621,"num_write_kb":18484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8155","reported_seq":8288,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.849578+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:06.849578+0000","last_peered":"2024-09-05T13:15:06.849578+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:06.849578+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":18440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3699},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7986","reported_seq":8121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.831428+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:06.831428+0000","last_peered":"2024-09-05T13:15:06.831428+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:06.831428+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7986,"log_dups_size":0,"ondisk_log_size":7986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32686080,"num_objects":7980,"num_object_clones":0,"num_object_copies":15960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":17220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3544},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8161","reported_seq":8285,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:06.830932+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:06.830932+0000","last_peered":"2024-09-05T13:15:06.830932+0000","last_clean":"2024-09-05T13:15:06.830932+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:06.830932+0000","last_undegraded":"2024-09-05T13:15:06.830932+0000","last_fullsized":"2024-09-05T13:15:06.830932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8161,"log_dups_size":0,"ondisk_log_size":8161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8054","reported_seq":8184,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.573044+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:07.573044+0000","last_peered":"2024-09-05T13:15:07.573044+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:07.573044+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8054,"log_dups_size":0,"ondisk_log_size":8054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5708,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3596}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8055","reported_seq":8188,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.791729+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:06.791729+0000","last_peered":"2024-09-05T13:15:06.791729+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:06.791729+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8055,"log_dups_size":0,"ondisk_log_size":8055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4422,"num_write_kb":17688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3593},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8075","reported_seq":13375,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.600419+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:07.600419+0000","last_peered":"2024-09-05T13:15:07.600419+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:07.600419+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8075,"log_dups_size":0,"ondisk_log_size":8075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5670,"num_write_kb":22680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2594,"num_bytes_recovered":10616832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1793},{"shards":"2","objects":6275}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7956","reported_seq":8074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.604676+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:07.604676+0000","last_peered":"2024-09-05T13:15:07.604676+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:07.604676+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7956,"log_dups_size":0,"ondisk_log_size":7956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5465,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8051","reported_seq":8186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.969097+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:06.969097+0000","last_peered":"2024-09-05T13:15:06.969097+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:06.969097+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32968704,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4202,"num_write_kb":16808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8124","reported_seq":8305,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.909070+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:06.909070+0000","last_peered":"2024-09-05T13:15:06.909070+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:06.909070+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8124,"log_dups_size":0,"ondisk_log_size":8124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243136,"num_objects":8116,"num_object_clones":0,"num_object_copies":16232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":16092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3672},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8207","reported_seq":8341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.848383+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:06.848383+0000","last_peered":"2024-09-05T13:15:06.848383+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:06.848383+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4337,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3637},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8218","reported_seq":8353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.870201+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:06.870201+0000","last_peered":"2024-09-05T13:15:06.870201+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:06.870201+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8218,"log_dups_size":0,"ondisk_log_size":8218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33612259,"num_objects":8207,"num_object_clones":0,"num_object_copies":16414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3642},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":529953251,"num_objects":129384,"num_object_clones":0,"num_object_copies":258768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":129384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85183,"num_write_kb":340732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2597,"num_bytes_recovered":10629120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1066852352,"data_stored":1066769253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129465,"ondisk_log_size":129465,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662332,"kb_used_data":401580,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93709508,"statfs":{"total":96636764160,"available":95958536192,"internally_reserved":0,"allocated":411217920,"data_stored":410972830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":266989229},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,0,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739628,"kb_used_data":459484,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93632212,"statfs":{"total":96636764160,"available":95879385088,"internally_reserved":0,"allocated":470511616,"data_stored":470262449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286846634},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343284,"kb_used_data":181092,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94028556,"statfs":{"total":96636764160,"available":96285241344,"internally_reserved":0,"allocated":185438208,"data_stored":185116781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":166063791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,8,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470380544,"data_stored":470376931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185155584,"data_stored":185079711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411316224,"data_stored":411312611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:09.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:10.231 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 157 20 130818 130798 3.25336 1.59766 0.056288 0.0240067 2024-09-05T13:15:10.346 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:10.347 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:10.369 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-05T13:15:09.562665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":529953270,"num_objects":129386,"num_object_clones":0,"num_object_copies":258772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":129386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85185,"num_write_kb":340734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2598,"num_bytes_recovered":10629139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129468,"ondisk_log_size":129468,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1745244,"kb_used_data":1042156,"kb_used_omap":12,"kb_used_meta":703027,"kb_avail":281370276,"statfs":{"total":289910292480,"available":288123162624,"internally_reserved":0,"allocated":1067167744,"data_stored":1066352060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":719899654},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,14,10,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23220224,"num_objects":5669,"num_object_clones":0,"num_object_copies":11338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-412,"num_objects_unfound":0,"num_objects_dirty":5669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5669,"num_write_kb":22676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1687552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004416"},"pg_stats":[{"pgid":"2.f","version":"43'8014","reported_seq":8084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.929435+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:06.929435+0000","last_peered":"2024-09-05T13:15:06.929435+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:06.929435+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8014,"num_write_kb":32056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3593},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8049","reported_seq":8149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:07.496008+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:07.496008+0000","last_peered":"2024-09-05T13:15:07.496008+0000","last_clean":"2024-09-05T13:15:07.496008+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:07.496008+0000","last_undegraded":"2024-09-05T13:15:07.496008+0000","last_fullsized":"2024-09-05T13:15:07.496008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6059,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8181","reported_seq":8304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.624380+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:07.624380+0000","last_peered":"2024-09-05T13:15:07.624380+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:07.624380+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8181,"log_dups_size":0,"ondisk_log_size":8181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33492992,"num_objects":8177,"num_object_clones":0,"num_object_copies":16354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5992,"num_write_kb":23968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3685}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8098","reported_seq":8161,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:07.630064+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:07.630064+0000","last_peered":"2024-09-05T13:15:07.630064+0000","last_clean":"2024-09-05T13:15:07.630064+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:07.630064+0000","last_undegraded":"2024-09-05T13:15:07.630064+0000","last_fullsized":"2024-09-05T13:15:07.630064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8098,"log_dups_size":0,"ondisk_log_size":8098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33169408,"num_objects":8098,"num_object_clones":0,"num_object_copies":16196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8098,"num_write_kb":32392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8081","reported_seq":8210,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:06.887480+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:06.887480+0000","last_peered":"2024-09-05T13:15:06.887480+0000","last_clean":"2024-09-05T13:15:06.887480+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:06.887480+0000","last_undegraded":"2024-09-05T13:15:06.887480+0000","last_fullsized":"2024-09-05T13:15:06.887480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8081,"log_dups_size":0,"ondisk_log_size":8081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33071104,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4621,"num_write_kb":18484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8155","reported_seq":8288,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.849578+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:06.849578+0000","last_peered":"2024-09-05T13:15:06.849578+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:06.849578+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":18440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3699},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7986","reported_seq":8121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.831428+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:06.831428+0000","last_peered":"2024-09-05T13:15:06.831428+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:06.831428+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7986,"log_dups_size":0,"ondisk_log_size":7986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32686080,"num_objects":7980,"num_object_clones":0,"num_object_copies":15960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":17220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3544},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8161","reported_seq":8285,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:06.830932+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:06.830932+0000","last_peered":"2024-09-05T13:15:06.830932+0000","last_clean":"2024-09-05T13:15:06.830932+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:06.830932+0000","last_undegraded":"2024-09-05T13:15:06.830932+0000","last_fullsized":"2024-09-05T13:15:06.830932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8161,"log_dups_size":0,"ondisk_log_size":8161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8054","reported_seq":8184,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.573044+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:07.573044+0000","last_peered":"2024-09-05T13:15:07.573044+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:07.573044+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8054,"log_dups_size":0,"ondisk_log_size":8054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5708,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3596}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8055","reported_seq":8188,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.791729+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:06.791729+0000","last_peered":"2024-09-05T13:15:06.791729+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:06.791729+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8055,"log_dups_size":0,"ondisk_log_size":8055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4422,"num_write_kb":17688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3593},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8075","reported_seq":13375,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.600419+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:07.600419+0000","last_peered":"2024-09-05T13:15:07.600419+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:07.600419+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8075,"log_dups_size":0,"ondisk_log_size":8075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5670,"num_write_kb":22680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2594,"num_bytes_recovered":10616832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1793},{"shards":"2","objects":6275}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7956","reported_seq":8074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.604676+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:07.604676+0000","last_peered":"2024-09-05T13:15:07.604676+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:07.604676+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7956,"log_dups_size":0,"ondisk_log_size":7956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5465,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8051","reported_seq":8186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.969097+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:06.969097+0000","last_peered":"2024-09-05T13:15:06.969097+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:06.969097+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32968704,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4202,"num_write_kb":16808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8124","reported_seq":8305,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.909070+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:06.909070+0000","last_peered":"2024-09-05T13:15:06.909070+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:06.909070+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8124,"log_dups_size":0,"ondisk_log_size":8124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243136,"num_objects":8116,"num_object_clones":0,"num_object_copies":16232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":16092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3672},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8207","reported_seq":8341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.848383+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:06.848383+0000","last_peered":"2024-09-05T13:15:06.848383+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:06.848383+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4337,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3637},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8218","reported_seq":8353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.870201+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:06.870201+0000","last_peered":"2024-09-05T13:15:06.870201+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:06.870201+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8218,"log_dups_size":0,"ondisk_log_size":8218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33612259,"num_objects":8207,"num_object_clones":0,"num_object_copies":16414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3642},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":529953251,"num_objects":129384,"num_object_clones":0,"num_object_copies":258768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":129384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85183,"num_write_kb":340732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2597,"num_bytes_recovered":10629120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1066852352,"data_stored":1066769253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129465,"ondisk_log_size":129465,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662332,"kb_used_data":401580,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93709508,"statfs":{"total":96636764160,"available":95958536192,"internally_reserved":0,"allocated":411217920,"data_stored":410972830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":266989229},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,0,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739628,"kb_used_data":459484,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93632212,"statfs":{"total":96636764160,"available":95879385088,"internally_reserved":0,"allocated":470511616,"data_stored":470262449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286846634},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343284,"kb_used_data":181092,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94028556,"statfs":{"total":96636764160,"available":96285241344,"internally_reserved":0,"allocated":185438208,"data_stored":185116781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":166063791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,8,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470380544,"data_stored":470376931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185155584,"data_stored":185079711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411316224,"data_stored":411312611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:10.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:10.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8014", 'reported_seq': 8084, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:06.929435+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:06.929435+0000', 'last_peered': '2024-09-05T13:15:06.929435+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:06.929435+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 8014, 'log_dups_size': 0, 'ondisk_log_size': 8014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32825344, 'num_objects': 8014, 'num_object_clones': 0, 'num_object_copies': 16028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8014, 'num_write_kb': 32056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3593}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8181", 'reported_seq': 8304, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:07.624380+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:07.624380+0000', 'last_peered': '2024-09-05T13:15:07.624380+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:07.624380+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 8181, 'log_dups_size': 0, 'ondisk_log_size': 8181, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33492992, 'num_objects': 8177, 'num_object_clones': 0, 'num_object_copies': 16354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5992, 'num_write_kb': 23968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 3685}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8155", 'reported_seq': 8288, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:06.849578+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:06.849578+0000', 'last_peered': '2024-09-05T13:15:06.849578+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:06.849578+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 8155, 'log_dups_size': 0, 'ondisk_log_size': 8155, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33382400, 'num_objects': 8150, 'num_object_clones': 0, 'num_object_copies': 16300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4610, 'num_write_kb': 18440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3699}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7986", 'reported_seq': 8121, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:06.831428+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:06.831428+0000', 'last_peered': '2024-09-05T13:15:06.831428+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:06.831428+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 7986, 'log_dups_size': 0, 'ondisk_log_size': 7986, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32686080, 'num_objects': 7980, 'num_object_clones': 0, 'num_object_copies': 15960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7980, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4305, 'num_write_kb': 17220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3544}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8054", 'reported_seq': 8184, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:07.573044+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:07.573044+0000', 'last_peered': '2024-09-05T13:15:07.573044+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:07.573044+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 8054, 'log_dups_size': 0, 'ondisk_log_size': 8054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32960512, 'num_objects': 8047, 'num_object_clones': 0, 'num_object_copies': 16094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5708, 'num_write_kb': 22832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 3596}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:10.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8055", 'reported_seq': 8188, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:06.791729+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:06.791729+0000', 'last_peered': '2024-09-05T13:15:06.791729+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:06.791729+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 8055, 'log_dups_size': 0, 'ondisk_log_size': 8055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32960512, 'num_objects': 8047, 'num_object_clones': 0, 'num_object_copies': 16094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4422, 'num_write_kb': 17688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3593}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8075", 'reported_seq': 13375, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:07.600419+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:07.600419+0000', 'last_peered': '2024-09-05T13:15:07.600419+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:07.600419+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 8075, 'log_dups_size': 0, 'ondisk_log_size': 8075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33054720, 'num_objects': 8070, 'num_object_clones': 0, 'num_object_copies': 16140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1793, 'num_objects_unfound': 0, 'num_objects_dirty': 8070, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5670, 'num_write_kb': 22680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2594, 'num_bytes_recovered': 10616832, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1793}, {'shards': '2', 'objects': 6275}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7956", 'reported_seq': 8074, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:07.604676+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:07.604676+0000', 'last_peered': '2024-09-05T13:15:07.604676+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:07.604676+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 7956, 'log_dups_size': 0, 'ondisk_log_size': 7956, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32563200, 'num_objects': 7950, 'num_object_clones': 0, 'num_object_copies': 15900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7950, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5465, 'num_write_kb': 21860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 3563}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8051", 'reported_seq': 8186, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:06.969097+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:06.969097+0000', 'last_peered': '2024-09-05T13:15:06.969097+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:06.969097+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 8051, 'log_dups_size': 0, 'ondisk_log_size': 8051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32968704, 'num_objects': 8049, 'num_object_clones': 0, 'num_object_copies': 16098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4202, 'num_write_kb': 16808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3630}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8124", 'reported_seq': 8305, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:06.909070+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:06.909070+0000', 'last_peered': '2024-09-05T13:15:06.909070+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:06.909070+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 8124, 'log_dups_size': 0, 'ondisk_log_size': 8124, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33243136, 'num_objects': 8116, 'num_object_clones': 0, 'num_object_copies': 16232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8116, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4023, 'num_write_kb': 16092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3672}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8207", 'reported_seq': 8341, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:06.848383+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:06.848383+0000', 'last_peered': '2024-09-05T13:15:06.848383+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:06.848383+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 8207, 'log_dups_size': 0, 'ondisk_log_size': 8207, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33599488, 'num_objects': 8203, 'num_object_clones': 0, 'num_object_copies': 16406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4337, 'num_write_kb': 17348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3637}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:10.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:10.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8218", 'reported_seq': 8353, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:06.870201+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:06.870201+0000', 'last_peered': '2024-09-05T13:15:06.870201+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:06.870201+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 8218, 'log_dups_size': 0, 'ondisk_log_size': 8218, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33612259, 'num_objects': 8207, 'num_object_clones': 0, 'num_object_copies': 16414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4886, 'num_write_kb': 19544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3642}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:10.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:11.077 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:11.077 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/258772 objects degraded (18.977%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":237,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129386,"data_bytes":529953270,"bytes_used":1787129856,"bytes_avail":288123162624,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":258772,"degraded_ratio":0.18977323667166462,"misplaced_objects":1793,"misplaced_total":258772,"misplaced_ratio":0.0069288794769140401,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":140577,"recovering_keys_per_sec":0,"num_objects_recovered":414,"num_bytes_recovered":1687552,"num_keys_recovered":0,"write_bytes_sec":1934306,"read_op_per_sec":0,"write_op_per_sec":472},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:11.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:11.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:11.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 158 20 131267 131247 3.24387 1.75391 0.0330606 0.0240762 2024-09-05T13:15:11.676 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:11.678 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:11.699 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-05T13:15:11.563302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":529953270,"num_objects":129386,"num_object_clones":0,"num_object_copies":258772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":129386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85185,"num_write_kb":340734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2598,"num_bytes_recovered":10629139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129468,"ondisk_log_size":129468,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1759948,"kb_used_data":1048540,"kb_used_omap":12,"kb_used_meta":711347,"kb_avail":281355572,"statfs":{"total":289910292480,"available":288108105728,"internally_reserved":0,"allocated":1073704960,"data_stored":1072889276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":728419334},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,6,7,13,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19644416,"num_objects":4796,"num_object_clones":0,"num_object_copies":9592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-269,"num_objects_unfound":0,"num_objects_dirty":4796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4796,"num_write_kb":19184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":1101824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004264"},"pg_stats":[{"pgid":"2.f","version":"43'8014","reported_seq":8084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.929435+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:06.929435+0000","last_peered":"2024-09-05T13:15:06.929435+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:06.929435+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8014,"log_dups_size":0,"ondisk_log_size":8014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8014,"num_write_kb":32056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3593},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8049","reported_seq":8149,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:07.496008+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:07.496008+0000","last_peered":"2024-09-05T13:15:07.496008+0000","last_clean":"2024-09-05T13:15:07.496008+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:07.496008+0000","last_undegraded":"2024-09-05T13:15:07.496008+0000","last_fullsized":"2024-09-05T13:15:07.496008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8049,"log_dups_size":0,"ondisk_log_size":8049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32944128,"num_objects":8043,"num_object_clones":0,"num_object_copies":16086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6059,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8181","reported_seq":8304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.624380+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:07.624380+0000","last_peered":"2024-09-05T13:15:07.624380+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:07.624380+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8181,"log_dups_size":0,"ondisk_log_size":8181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33492992,"num_objects":8177,"num_object_clones":0,"num_object_copies":16354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5992,"num_write_kb":23968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3685}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8098","reported_seq":8161,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:07.630064+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:07.630064+0000","last_peered":"2024-09-05T13:15:07.630064+0000","last_clean":"2024-09-05T13:15:07.630064+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:07.630064+0000","last_undegraded":"2024-09-05T13:15:07.630064+0000","last_fullsized":"2024-09-05T13:15:07.630064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8098,"log_dups_size":0,"ondisk_log_size":8098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33169408,"num_objects":8098,"num_object_clones":0,"num_object_copies":16196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8098,"num_write_kb":32392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8081","reported_seq":8210,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:06.887480+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:06.887480+0000","last_peered":"2024-09-05T13:15:06.887480+0000","last_clean":"2024-09-05T13:15:06.887480+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:06.887480+0000","last_undegraded":"2024-09-05T13:15:06.887480+0000","last_fullsized":"2024-09-05T13:15:06.887480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8081,"log_dups_size":0,"ondisk_log_size":8081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33071104,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4621,"num_write_kb":18484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8155","reported_seq":8288,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.849578+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:06.849578+0000","last_peered":"2024-09-05T13:15:06.849578+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:06.849578+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":18440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3699},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7986","reported_seq":8121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.831428+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:06.831428+0000","last_peered":"2024-09-05T13:15:06.831428+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:06.831428+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":7986,"log_dups_size":0,"ondisk_log_size":7986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32686080,"num_objects":7980,"num_object_clones":0,"num_object_copies":15960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":17220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3544},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8161","reported_seq":8285,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:06.830932+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:06.830932+0000","last_peered":"2024-09-05T13:15:06.830932+0000","last_clean":"2024-09-05T13:15:06.830932+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:06.830932+0000","last_undegraded":"2024-09-05T13:15:06.830932+0000","last_fullsized":"2024-09-05T13:15:06.830932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8161,"log_dups_size":0,"ondisk_log_size":8161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8054","reported_seq":8184,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.573044+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:07.573044+0000","last_peered":"2024-09-05T13:15:07.573044+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:07.573044+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8054,"log_dups_size":0,"ondisk_log_size":8054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5708,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3596}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8055","reported_seq":8188,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.791729+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:06.791729+0000","last_peered":"2024-09-05T13:15:06.791729+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:06.791729+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8055,"log_dups_size":0,"ondisk_log_size":8055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4422,"num_write_kb":17688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3593},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8075","reported_seq":13375,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.600419+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:07.600419+0000","last_peered":"2024-09-05T13:15:07.600419+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:07.600419+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8075,"log_dups_size":0,"ondisk_log_size":8075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5670,"num_write_kb":22680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2594,"num_bytes_recovered":10616832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1793},{"shards":"2","objects":6275}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7956","reported_seq":8074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:07.604676+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:07.604676+0000","last_peered":"2024-09-05T13:15:07.604676+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:07.604676+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":7956,"log_dups_size":0,"ondisk_log_size":7956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5465,"num_write_kb":21860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3563}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8051","reported_seq":8186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.969097+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:06.969097+0000","last_peered":"2024-09-05T13:15:06.969097+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:06.969097+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32968704,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4202,"num_write_kb":16808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8124","reported_seq":8305,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.909070+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:06.909070+0000","last_peered":"2024-09-05T13:15:06.909070+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:06.909070+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8124,"log_dups_size":0,"ondisk_log_size":8124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243136,"num_objects":8116,"num_object_clones":0,"num_object_copies":16232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4023,"num_write_kb":16092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3672},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8207","reported_seq":8341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.848383+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:06.848383+0000","last_peered":"2024-09-05T13:15:06.848383+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:06.848383+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4337,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3637},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8218","reported_seq":8353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:06.870201+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:06.870201+0000","last_peered":"2024-09-05T13:15:06.870201+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:06.870201+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8218,"log_dups_size":0,"ondisk_log_size":8218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33612259,"num_objects":8207,"num_object_clones":0,"num_object_copies":16414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3642},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":529953251,"num_objects":129384,"num_object_clones":0,"num_object_copies":258768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1793,"num_objects_unfound":0,"num_objects_dirty":129384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85183,"num_write_kb":340732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2597,"num_bytes_recovered":10629120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1074864128,"data_stored":1074781029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129465,"ondisk_log_size":129465,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662332,"kb_used_data":401580,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93709508,"statfs":{"total":96636764160,"available":95958536192,"internally_reserved":0,"allocated":411217920,"data_stored":410972830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":266989229},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,0,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739628,"kb_used_data":459484,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93632212,"statfs":{"total":96636764160,"available":95879385088,"internally_reserved":0,"allocated":470511616,"data_stored":470262449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286846634},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357988,"kb_used_data":187476,"kb_used_omap":4,"kb_used_meta":170491,"kb_avail":94013852,"statfs":{"total":96636764160,"available":96270184448,"internally_reserved":0,"allocated":191975424,"data_stored":191653997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":174583471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470380544,"data_stored":470376931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":193167360,"data_stored":193091487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411316224,"data_stored":411312611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:12.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 159 20 131746 131726 3.23524 1.87109 0.0240153 0.024141 2024-09-05T13:15:13.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:15:13.231837+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0242004 2024-09-05T13:15:13.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:15:13.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 160 20 132246 132226 3.22723 1.95312 0.0425433 0.0242004 2024-09-05T13:15:14.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 161 20 132741 132721 3.2192 1.93359 0.0117835 0.0242603 2024-09-05T13:15:14.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:15.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 162 20 133208 133188 3.21059 1.82422 0.019546 0.0243253 2024-09-05T13:15:15.280 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:15.281 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:15.307 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-05T13:15:13.564109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":539632118,"num_objects":131749,"num_object_clones":0,"num_object_copies":263498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":131749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87548,"num_write_kb":350186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2726,"num_bytes_recovered":11153427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131831,"ondisk_log_size":131831,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1763236,"kb_used_data":1060148,"kb_used_omap":12,"kb_used_meta":703027,"kb_avail":281352284,"statfs":{"total":289910292480,"available":288104738816,"internally_reserved":0,"allocated":1085591552,"data_stored":1084775868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":719899654},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,2,6,13,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29323264,"num_objects":7159,"num_object_clones":0,"num_object_copies":14318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-397,"num_objects_unfound":0,"num_objects_dirty":7159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7159,"num_write_kb":28636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":397,"num_bytes_recovered":1626112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004389"},"pg_stats":[{"pgid":"2.f","version":"43'8159","reported_seq":8229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.946096+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:11.946096+0000","last_peered":"2024-09-05T13:15:11.946096+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:11.946096+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8159,"log_dups_size":0,"ondisk_log_size":8159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8159,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3738},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8201","reported_seq":8301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:12.598284+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:12.598284+0000","last_peered":"2024-09-05T13:15:12.598284+0000","last_clean":"2024-09-05T13:15:12.598284+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:12.598284+0000","last_undegraded":"2024-09-05T13:15:12.598284+0000","last_fullsized":"2024-09-05T13:15:12.598284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8201,"log_dups_size":0,"ondisk_log_size":8201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6211,"num_write_kb":24844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8317","reported_seq":8440,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.591572+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:12.591572+0000","last_peered":"2024-09-05T13:15:12.591572+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:12.591572+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8317,"log_dups_size":0,"ondisk_log_size":8317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34050048,"num_objects":8313,"num_object_clones":0,"num_object_copies":16626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6128,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3821}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8234","reported_seq":8299,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:12.540689+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:12.540689+0000","last_peered":"2024-09-05T13:15:12.540689+0000","last_clean":"2024-09-05T13:15:12.540689+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:12.540689+0000","last_undegraded":"2024-09-05T13:15:12.540689+0000","last_fullsized":"2024-09-05T13:15:12.540689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8234,"log_dups_size":0,"ondisk_log_size":8234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33726464,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8234,"num_write_kb":32936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8240","reported_seq":8369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:11.984841+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:11.984841+0000","last_peered":"2024-09-05T13:15:11.984841+0000","last_clean":"2024-09-05T13:15:11.984841+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:11.984841+0000","last_undegraded":"2024-09-05T13:15:11.984841+0000","last_fullsized":"2024-09-05T13:15:11.984841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8240,"log_dups_size":0,"ondisk_log_size":8240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33722368,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4780,"num_write_kb":19120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8303","reported_seq":8435,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.997107+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:11.997107+0000","last_peered":"2024-09-05T13:15:11.997107+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:11.997107+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8303,"log_dups_size":0,"ondisk_log_size":8303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4758,"num_write_kb":19032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8143","reported_seq":8278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.967108+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:11.967108+0000","last_peered":"2024-09-05T13:15:11.967108+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:11.967108+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8143,"log_dups_size":0,"ondisk_log_size":8143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33329152,"num_objects":8137,"num_object_clones":0,"num_object_copies":16274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4462,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3701},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8307","reported_seq":8434,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:11.960734+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:11.960734+0000","last_peered":"2024-09-05T13:15:11.960734+0000","last_clean":"2024-09-05T13:15:11.960734+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:11.960734+0000","last_undegraded":"2024-09-05T13:15:11.960734+0000","last_fullsized":"2024-09-05T13:15:11.960734+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34017280,"num_objects":8305,"num_object_clones":0,"num_object_copies":16610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.912421+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.912421+0000","last_peered":"2024-09-05T13:13:17.912421+0000","last_clean":"2024-09-05T13:13:17.912421+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.912421+0000","last_undegraded":"2024-09-05T13:13:17.912421+0000","last_fullsized":"2024-09-05T13:13:17.912421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8208","reported_seq":8338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.565442+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:12.565442+0000","last_peered":"2024-09-05T13:15:12.565442+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:12.565442+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8208,"log_dups_size":0,"ondisk_log_size":8208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33591296,"num_objects":8201,"num_object_clones":0,"num_object_copies":16402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5862,"num_write_kb":23448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3750}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8211","reported_seq":8344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.963930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:11.963930+0000","last_peered":"2024-09-05T13:15:11.963930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:11.963930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8211,"log_dups_size":0,"ondisk_log_size":8211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4578,"num_write_kb":18312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3749},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8228","reported_seq":13784,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.633090+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:12.633090+0000","last_peered":"2024-09-05T13:15:12.633090+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:12.633090+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8228,"log_dups_size":0,"ondisk_log_size":8228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2722,"num_bytes_recovered":11141120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1665},{"shards":"2","objects":6556}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8085","reported_seq":8203,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.452907+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:12.452907+0000","last_peered":"2024-09-05T13:15:12.452907+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:12.452907+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5594,"num_write_kb":22376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8208","reported_seq":8343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.885005+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:11.885005+0000","last_peered":"2024-09-05T13:15:11.885005+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:11.885005+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8208,"log_dups_size":0,"ondisk_log_size":8208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33611776,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3787},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8265","reported_seq":8447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.978875+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:11.978875+0000","last_peered":"2024-09-05T13:15:11.978875+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:11.978875+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8265,"log_dups_size":0,"ondisk_log_size":8265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33820672,"num_objects":8257,"num_object_clones":0,"num_object_copies":16514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4164,"num_write_kb":16656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8346","reported_seq":8480,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.938153+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:11.938153+0000","last_peered":"2024-09-05T13:15:11.938153+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:11.938153+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34168832,"num_objects":8342,"num_object_clones":0,"num_object_copies":16684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4476,"num_write_kb":17904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3776},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8373","reported_seq":8506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.967207+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:11.967207+0000","last_peered":"2024-09-05T13:15:11.967207+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:11.967207+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8373,"log_dups_size":0,"ondisk_log_size":8373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34247139,"num_objects":8362,"num_object_clones":0,"num_object_copies":16724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5041,"num_write_kb":20164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":539632099,"num_objects":131747,"num_object_clones":0,"num_object_copies":263494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":131747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87546,"num_write_kb":350184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2725,"num_bytes_recovered":11153408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1086853120,"data_stored":1086770021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131828,"ondisk_log_size":131828,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":654524,"kb_used_data":406252,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93717316,"statfs":{"total":96636764160,"available":95966531584,"internally_reserved":0,"allocated":416002048,"data_stored":415756958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750724,"kb_used_data":466420,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93621116,"statfs":{"total":96636764160,"available":95868022784,"internally_reserved":0,"allocated":477614080,"data_stored":477364913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,4,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357988,"kb_used_data":187476,"kb_used_omap":4,"kb_used_meta":170491,"kb_avail":94013852,"statfs":{"total":96636764160,"available":96270184448,"internally_reserved":0,"allocated":191975424,"data_stored":191653997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":174583471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477618176,"data_stored":477614563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":193167360,"data_stored":193091487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":416067584,"data_stored":416063971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:15.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:15.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:15.859 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:15.881 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-05T13:15:15.564913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":539632118,"num_objects":131749,"num_object_clones":0,"num_object_copies":263498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":131749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87548,"num_write_kb":350186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2726,"num_bytes_recovered":11153427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131831,"ondisk_log_size":131831,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1777968,"kb_used_data":1069312,"kb_used_omap":12,"kb_used_meta":708595,"kb_avail":281337552,"statfs":{"total":289910292480,"available":288089653248,"internally_reserved":0,"allocated":1094975488,"data_stored":1094159804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":725601286},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,2,9,9,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19456000,"num_objects":4750,"num_object_clones":0,"num_object_copies":9500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":4750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":19000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004518"},"pg_stats":[{"pgid":"2.f","version":"43'8159","reported_seq":8229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.946096+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:11.946096+0000","last_peered":"2024-09-05T13:15:11.946096+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:11.946096+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8159,"log_dups_size":0,"ondisk_log_size":8159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8159,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3738},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8201","reported_seq":8301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:12.598284+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:12.598284+0000","last_peered":"2024-09-05T13:15:12.598284+0000","last_clean":"2024-09-05T13:15:12.598284+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:12.598284+0000","last_undegraded":"2024-09-05T13:15:12.598284+0000","last_fullsized":"2024-09-05T13:15:12.598284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8201,"log_dups_size":0,"ondisk_log_size":8201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6211,"num_write_kb":24844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8317","reported_seq":8440,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.591572+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:12.591572+0000","last_peered":"2024-09-05T13:15:12.591572+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:12.591572+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8317,"log_dups_size":0,"ondisk_log_size":8317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34050048,"num_objects":8313,"num_object_clones":0,"num_object_copies":16626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6128,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3821}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8234","reported_seq":8299,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:12.540689+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:12.540689+0000","last_peered":"2024-09-05T13:15:12.540689+0000","last_clean":"2024-09-05T13:15:12.540689+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:12.540689+0000","last_undegraded":"2024-09-05T13:15:12.540689+0000","last_fullsized":"2024-09-05T13:15:12.540689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8234,"log_dups_size":0,"ondisk_log_size":8234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33726464,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8234,"num_write_kb":32936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8240","reported_seq":8369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:11.984841+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:11.984841+0000","last_peered":"2024-09-05T13:15:11.984841+0000","last_clean":"2024-09-05T13:15:11.984841+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:11.984841+0000","last_undegraded":"2024-09-05T13:15:11.984841+0000","last_fullsized":"2024-09-05T13:15:11.984841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8240,"log_dups_size":0,"ondisk_log_size":8240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33722368,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4780,"num_write_kb":19120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8303","reported_seq":8435,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.997107+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:11.997107+0000","last_peered":"2024-09-05T13:15:11.997107+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:11.997107+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8303,"log_dups_size":0,"ondisk_log_size":8303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4758,"num_write_kb":19032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8143","reported_seq":8278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.967108+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:11.967108+0000","last_peered":"2024-09-05T13:15:11.967108+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:11.967108+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8143,"log_dups_size":0,"ondisk_log_size":8143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33329152,"num_objects":8137,"num_object_clones":0,"num_object_copies":16274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4462,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3701},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8307","reported_seq":8434,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:11.960734+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:11.960734+0000","last_peered":"2024-09-05T13:15:11.960734+0000","last_clean":"2024-09-05T13:15:11.960734+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:11.960734+0000","last_undegraded":"2024-09-05T13:15:11.960734+0000","last_fullsized":"2024-09-05T13:15:11.960734+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34017280,"num_objects":8305,"num_object_clones":0,"num_object_copies":16610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8208","reported_seq":8338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.565442+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:12.565442+0000","last_peered":"2024-09-05T13:15:12.565442+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:12.565442+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8208,"log_dups_size":0,"ondisk_log_size":8208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33591296,"num_objects":8201,"num_object_clones":0,"num_object_copies":16402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5862,"num_write_kb":23448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3750}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8211","reported_seq":8344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.963930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:11.963930+0000","last_peered":"2024-09-05T13:15:11.963930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:11.963930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8211,"log_dups_size":0,"ondisk_log_size":8211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4578,"num_write_kb":18312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3749},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8228","reported_seq":13784,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.633090+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:12.633090+0000","last_peered":"2024-09-05T13:15:12.633090+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:12.633090+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8228,"log_dups_size":0,"ondisk_log_size":8228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2722,"num_bytes_recovered":11141120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1665},{"shards":"2","objects":6556}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8085","reported_seq":8203,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.452907+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:12.452907+0000","last_peered":"2024-09-05T13:15:12.452907+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:12.452907+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5594,"num_write_kb":22376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8208","reported_seq":8343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.885005+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:11.885005+0000","last_peered":"2024-09-05T13:15:11.885005+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:11.885005+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8208,"log_dups_size":0,"ondisk_log_size":8208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33611776,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3787},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8265","reported_seq":8447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.978875+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:11.978875+0000","last_peered":"2024-09-05T13:15:11.978875+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:11.978875+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8265,"log_dups_size":0,"ondisk_log_size":8265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33820672,"num_objects":8257,"num_object_clones":0,"num_object_copies":16514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4164,"num_write_kb":16656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8346","reported_seq":8480,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.938153+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:11.938153+0000","last_peered":"2024-09-05T13:15:11.938153+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:11.938153+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34168832,"num_objects":8342,"num_object_clones":0,"num_object_copies":16684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4476,"num_write_kb":17904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3776},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8373","reported_seq":8506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.967207+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:11.967207+0000","last_peered":"2024-09-05T13:15:11.967207+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:11.967207+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8373,"log_dups_size":0,"ondisk_log_size":8373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34247139,"num_objects":8362,"num_object_clones":0,"num_object_copies":16724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5041,"num_write_kb":20164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":539632099,"num_objects":131747,"num_object_clones":0,"num_object_copies":263494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":131747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87546,"num_write_kb":350184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2725,"num_bytes_recovered":11153408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1094688768,"data_stored":1094605669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131828,"ondisk_log_size":131828,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":654524,"kb_used_data":406252,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93717316,"statfs":{"total":96636764160,"available":95966531584,"internally_reserved":0,"allocated":416002048,"data_stored":415756958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750724,"kb_used_data":466420,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93621116,"statfs":{"total":96636764160,"available":95868022784,"internally_reserved":0,"allocated":477614080,"data_stored":477364913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,4,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":372720,"kb_used_data":196640,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93999120,"statfs":{"total":96636764160,"available":96255098880,"internally_reserved":0,"allocated":201359360,"data_stored":201037933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477618176,"data_stored":477614563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":201003008,"data_stored":200927135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":416067584,"data_stored":416063971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:15.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:15.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8159", 'reported_seq': 8229, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:11.946096+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:11.946096+0000', 'last_peered': '2024-09-05T13:15:11.946096+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:11.946096+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 8159, 'log_dups_size': 0, 'ondisk_log_size': 8159, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33419264, 'num_objects': 8159, 'num_object_clones': 0, 'num_object_copies': 16318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8159, 'num_write_kb': 32636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3738}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8317", 'reported_seq': 8440, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:12.591572+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:12.591572+0000', 'last_peered': '2024-09-05T13:15:12.591572+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:12.591572+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 8317, 'log_dups_size': 0, 'ondisk_log_size': 8317, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34050048, 'num_objects': 8313, 'num_object_clones': 0, 'num_object_copies': 16626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6128, 'num_write_kb': 24512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 3821}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8303", 'reported_seq': 8435, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:11.997107+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:11.997107+0000', 'last_peered': '2024-09-05T13:15:11.997107+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:11.997107+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 8303, 'log_dups_size': 0, 'ondisk_log_size': 8303, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33988608, 'num_objects': 8298, 'num_object_clones': 0, 'num_object_copies': 16596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4758, 'num_write_kb': 19032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3847}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8143", 'reported_seq': 8278, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:11.967108+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:11.967108+0000', 'last_peered': '2024-09-05T13:15:11.967108+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:11.967108+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 8143, 'log_dups_size': 0, 'ondisk_log_size': 8143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33329152, 'num_objects': 8137, 'num_object_clones': 0, 'num_object_copies': 16274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4462, 'num_write_kb': 17848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3701}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8208", 'reported_seq': 8338, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:12.565442+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:12.565442+0000', 'last_peered': '2024-09-05T13:15:12.565442+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:12.565442+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 8208, 'log_dups_size': 0, 'ondisk_log_size': 8208, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33591296, 'num_objects': 8201, 'num_object_clones': 0, 'num_object_copies': 16402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5862, 'num_write_kb': 23448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 3750}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:15.883 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8211", 'reported_seq': 8344, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:11.963930+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:11.963930+0000', 'last_peered': '2024-09-05T13:15:11.963930+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:11.963930+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 8211, 'log_dups_size': 0, 'ondisk_log_size': 8211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33599488, 'num_objects': 8203, 'num_object_clones': 0, 'num_object_copies': 16406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4578, 'num_write_kb': 18312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3749}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8228", 'reported_seq': 13784, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:12.633090+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:12.633090+0000', 'last_peered': '2024-09-05T13:15:12.633090+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:12.633090+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 8228, 'log_dups_size': 0, 'ondisk_log_size': 8228, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33681408, 'num_objects': 8223, 'num_object_clones': 0, 'num_object_copies': 16446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1665, 'num_objects_unfound': 0, 'num_objects_dirty': 8223, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5823, 'num_write_kb': 23292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2722, 'num_bytes_recovered': 11141120, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1665}, {'shards': '2', 'objects': 6556}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8085", 'reported_seq': 8203, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:12.452907+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:12.452907+0000', 'last_peered': '2024-09-05T13:15:12.452907+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:12.452907+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 8085, 'log_dups_size': 0, 'ondisk_log_size': 8085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33091584, 'num_objects': 8079, 'num_object_clones': 0, 'num_object_copies': 16158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8079, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5594, 'num_write_kb': 22376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 3692}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8208", 'reported_seq': 8343, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:11.885005+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:11.885005+0000', 'last_peered': '2024-09-05T13:15:11.885005+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:11.885005+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 8208, 'log_dups_size': 0, 'ondisk_log_size': 8208, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33611776, 'num_objects': 8206, 'num_object_clones': 0, 'num_object_copies': 16412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8206, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4359, 'num_write_kb': 17436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3787}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8265", 'reported_seq': 8447, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:11.978875+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:11.978875+0000', 'last_peered': '2024-09-05T13:15:11.978875+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:11.978875+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 8265, 'log_dups_size': 0, 'ondisk_log_size': 8265, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33820672, 'num_objects': 8257, 'num_object_clones': 0, 'num_object_copies': 16514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8257, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4164, 'num_write_kb': 16656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3813}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:15.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:15.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8346", 'reported_seq': 8480, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:11.938153+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:11.938153+0000', 'last_peered': '2024-09-05T13:15:11.938153+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:11.938153+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 8346, 'log_dups_size': 0, 'ondisk_log_size': 8346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34168832, 'num_objects': 8342, 'num_object_clones': 0, 'num_object_copies': 16684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4476, 'num_write_kb': 17904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3776}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:15.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:15.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8373", 'reported_seq': 8506, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:11.967207+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:11.967207+0000', 'last_peered': '2024-09-05T13:15:11.967207+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:11.967207+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 8373, 'log_dups_size': 0, 'ondisk_log_size': 8373, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34247139, 'num_objects': 8362, 'num_object_clones': 0, 'num_object_copies': 16724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8362, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5041, 'num_write_kb': 20164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3797}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:15.885 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:16.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 163 20 133639 133619 3.20122 1.68359 0.0350177 0.0243957 2024-09-05T13:15:16.580 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:16.581 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/263498 objects degraded (18.637%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":243,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":131749,"data_bytes":539632118,"bytes_used":1820639232,"bytes_avail":288089653248,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":263498,"degraded_ratio":0.18636953601165854,"misplaced_objects":1665,"misplaced_total":263498,"misplaced_ratio":0.0063188335395335075,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":90760,"recovering_keys_per_sec":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"write_bytes_sec":1620723,"read_op_per_sec":0,"write_op_per_sec":395},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:16.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:16.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:17.186 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:17.187 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:17.223 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-05T13:15:15.564913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":539632118,"num_objects":131749,"num_object_clones":0,"num_object_copies":263498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":131749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87548,"num_write_kb":350186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2726,"num_bytes_recovered":11153427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131831,"ondisk_log_size":131831,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1777968,"kb_used_data":1069312,"kb_used_omap":12,"kb_used_meta":708595,"kb_avail":281337552,"statfs":{"total":289910292480,"available":288089653248,"internally_reserved":0,"allocated":1094975488,"data_stored":1094159804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":725601286},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,2,9,9,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19456000,"num_objects":4750,"num_object_clones":0,"num_object_copies":9500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":4750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4750,"num_write_kb":19000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004518"},"pg_stats":[{"pgid":"2.f","version":"43'8159","reported_seq":8229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.946096+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:11.946096+0000","last_peered":"2024-09-05T13:15:11.946096+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:11.946096+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8159,"log_dups_size":0,"ondisk_log_size":8159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33419264,"num_objects":8159,"num_object_clones":0,"num_object_copies":16318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8159,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3738},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8201","reported_seq":8301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:12.598284+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:12.598284+0000","last_peered":"2024-09-05T13:15:12.598284+0000","last_clean":"2024-09-05T13:15:12.598284+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:12.598284+0000","last_undegraded":"2024-09-05T13:15:12.598284+0000","last_fullsized":"2024-09-05T13:15:12.598284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8201,"log_dups_size":0,"ondisk_log_size":8201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6211,"num_write_kb":24844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8317","reported_seq":8440,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.591572+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:12.591572+0000","last_peered":"2024-09-05T13:15:12.591572+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:12.591572+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8317,"log_dups_size":0,"ondisk_log_size":8317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34050048,"num_objects":8313,"num_object_clones":0,"num_object_copies":16626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6128,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3821}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8234","reported_seq":8299,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:12.540689+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:12.540689+0000","last_peered":"2024-09-05T13:15:12.540689+0000","last_clean":"2024-09-05T13:15:12.540689+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:12.540689+0000","last_undegraded":"2024-09-05T13:15:12.540689+0000","last_fullsized":"2024-09-05T13:15:12.540689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8234,"log_dups_size":0,"ondisk_log_size":8234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33726464,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8234,"num_write_kb":32936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8240","reported_seq":8369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:11.984841+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:11.984841+0000","last_peered":"2024-09-05T13:15:11.984841+0000","last_clean":"2024-09-05T13:15:11.984841+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:11.984841+0000","last_undegraded":"2024-09-05T13:15:11.984841+0000","last_fullsized":"2024-09-05T13:15:11.984841+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8240,"log_dups_size":0,"ondisk_log_size":8240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33722368,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4780,"num_write_kb":19120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8303","reported_seq":8435,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.997107+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:11.997107+0000","last_peered":"2024-09-05T13:15:11.997107+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:11.997107+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8303,"log_dups_size":0,"ondisk_log_size":8303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4758,"num_write_kb":19032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8143","reported_seq":8278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.967108+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:11.967108+0000","last_peered":"2024-09-05T13:15:11.967108+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:11.967108+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8143,"log_dups_size":0,"ondisk_log_size":8143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33329152,"num_objects":8137,"num_object_clones":0,"num_object_copies":16274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4462,"num_write_kb":17848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3701},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8307","reported_seq":8434,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:11.960734+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:11.960734+0000","last_peered":"2024-09-05T13:15:11.960734+0000","last_clean":"2024-09-05T13:15:11.960734+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:11.960734+0000","last_undegraded":"2024-09-05T13:15:11.960734+0000","last_fullsized":"2024-09-05T13:15:11.960734+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34017280,"num_objects":8305,"num_object_clones":0,"num_object_copies":16610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8208","reported_seq":8338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.565442+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:12.565442+0000","last_peered":"2024-09-05T13:15:12.565442+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:12.565442+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8208,"log_dups_size":0,"ondisk_log_size":8208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33591296,"num_objects":8201,"num_object_clones":0,"num_object_copies":16402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5862,"num_write_kb":23448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3750}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8211","reported_seq":8344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.963930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:11.963930+0000","last_peered":"2024-09-05T13:15:11.963930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:11.963930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8211,"log_dups_size":0,"ondisk_log_size":8211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4578,"num_write_kb":18312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3749},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8228","reported_seq":13784,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.633090+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:12.633090+0000","last_peered":"2024-09-05T13:15:12.633090+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:12.633090+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8228,"log_dups_size":0,"ondisk_log_size":8228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2722,"num_bytes_recovered":11141120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1665},{"shards":"2","objects":6556}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8085","reported_seq":8203,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:12.452907+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:12.452907+0000","last_peered":"2024-09-05T13:15:12.452907+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:12.452907+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8085,"log_dups_size":0,"ondisk_log_size":8085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5594,"num_write_kb":22376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8208","reported_seq":8343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.885005+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:11.885005+0000","last_peered":"2024-09-05T13:15:11.885005+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:11.885005+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8208,"log_dups_size":0,"ondisk_log_size":8208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33611776,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3787},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8265","reported_seq":8447,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.978875+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:11.978875+0000","last_peered":"2024-09-05T13:15:11.978875+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:11.978875+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8265,"log_dups_size":0,"ondisk_log_size":8265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33820672,"num_objects":8257,"num_object_clones":0,"num_object_copies":16514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4164,"num_write_kb":16656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8346","reported_seq":8480,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.938153+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:11.938153+0000","last_peered":"2024-09-05T13:15:11.938153+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:11.938153+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34168832,"num_objects":8342,"num_object_clones":0,"num_object_copies":16684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4476,"num_write_kb":17904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3776},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8373","reported_seq":8506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:11.967207+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:11.967207+0000","last_peered":"2024-09-05T13:15:11.967207+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:11.967207+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8373,"log_dups_size":0,"ondisk_log_size":8373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34247139,"num_objects":8362,"num_object_clones":0,"num_object_copies":16724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5041,"num_write_kb":20164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":539632099,"num_objects":131747,"num_object_clones":0,"num_object_copies":263494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1665,"num_objects_unfound":0,"num_objects_dirty":131747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87546,"num_write_kb":350184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2725,"num_bytes_recovered":11153408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1094688768,"data_stored":1094605669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131828,"ondisk_log_size":131828,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":654524,"kb_used_data":406252,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93717316,"statfs":{"total":96636764160,"available":95966531584,"internally_reserved":0,"allocated":416002048,"data_stored":415756958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750724,"kb_used_data":466420,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93621116,"statfs":{"total":96636764160,"available":95868022784,"internally_reserved":0,"allocated":477614080,"data_stored":477364913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,4,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":372720,"kb_used_data":196640,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93999120,"statfs":{"total":96636764160,"available":96255098880,"internally_reserved":0,"allocated":201359360,"data_stored":201037933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477618176,"data_stored":477614563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":201003008,"data_stored":200927135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":416067584,"data_stored":416063971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:17.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 164 20 134079 134059 3.19218 1.71875 0.111024 0.0244651 2024-09-05T13:15:18.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 165 20 134573 134553 3.18453 1.92969 0.0139194 0.0245245 2024-09-05T13:15:19.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 166 20 135077 135057 3.17721 1.96875 0.0675808 0.0245821 2024-09-05T13:15:20.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:20.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 167 20 135571 135551 3.16974 1.92969 0.0521092 0.0246403 2024-09-05T13:15:20.769 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:20.772 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:20.798 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-05T13:15:19.566393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549310966,"num_objects":134112,"num_object_clones":0,"num_object_copies":268224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":134112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89911,"num_write_kb":359638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2872,"num_bytes_recovered":11751443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134194,"ondisk_log_size":134194,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1820144,"kb_used_data":1080960,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281295376,"statfs":{"total":289910292480,"available":288046465024,"internally_reserved":0,"allocated":1106903040,"data_stored":1106087356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":756861958},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,6,8,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-412,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5581,"num_write_kb":22324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":1687552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004412"},"pg_stats":[{"pgid":"2.f","version":"43'8302","reported_seq":8372,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.061115+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:17.061115+0000","last_peered":"2024-09-05T13:15:17.061115+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:17.061115+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8302,"log_dups_size":0,"ondisk_log_size":8302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8302,"num_write_kb":33208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3881},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8326","reported_seq":8424,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.623671+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:17.623671+0000","last_peered":"2024-09-05T13:15:17.623671+0000","last_clean":"2024-09-05T13:15:17.623671+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:17.623671+0000","last_undegraded":"2024-09-05T13:15:17.623671+0000","last_fullsized":"2024-09-05T13:15:17.623671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8326,"log_dups_size":0,"ondisk_log_size":8326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34078720,"num_objects":8320,"num_object_clones":0,"num_object_copies":16640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6336,"num_write_kb":25344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8459","reported_seq":8582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.541561+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:17.541561+0000","last_peered":"2024-09-05T13:15:17.541561+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:17.541561+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8459,"log_dups_size":0,"ondisk_log_size":8459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34631680,"num_objects":8455,"num_object_clones":0,"num_object_copies":16910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6270,"num_write_kb":25080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8384","reported_seq":8449,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.624120+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:17.624120+0000","last_peered":"2024-09-05T13:15:17.624120+0000","last_clean":"2024-09-05T13:15:17.624120+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:17.624120+0000","last_undegraded":"2024-09-05T13:15:17.624120+0000","last_fullsized":"2024-09-05T13:15:17.624120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8384,"log_dups_size":0,"ondisk_log_size":8384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":33536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8380","reported_seq":8509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.027577+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:17.027577+0000","last_peered":"2024-09-05T13:15:17.027577+0000","last_clean":"2024-09-05T13:15:17.027577+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:17.027577+0000","last_undegraded":"2024-09-05T13:15:17.027577+0000","last_fullsized":"2024-09-05T13:15:17.027577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4920,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8482","reported_seq":8613,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.046704+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:17.046704+0000","last_peered":"2024-09-05T13:15:17.046704+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:17.046704+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8482,"log_dups_size":0,"ondisk_log_size":8482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4937,"num_write_kb":19748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8293","reported_seq":8428,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:16.966121+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:16.966121+0000","last_peered":"2024-09-05T13:15:16.966121+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:16.966121+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8293,"log_dups_size":0,"ondisk_log_size":8293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33943552,"num_objects":8287,"num_object_clones":0,"num_object_copies":16574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4612,"num_write_kb":18448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3851},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8469","reported_seq":8596,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:16.985146+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:16.985146+0000","last_peered":"2024-09-05T13:15:16.985146+0000","last_clean":"2024-09-05T13:15:16.985146+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:16.985146+0000","last_undegraded":"2024-09-05T13:15:16.985146+0000","last_fullsized":"2024-09-05T13:15:16.985146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8469,"log_dups_size":0,"ondisk_log_size":8469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34680832,"num_objects":8467,"num_object_clones":0,"num_object_copies":16934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5079,"num_write_kb":20316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8372","reported_seq":8502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.613359+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:17.613359+0000","last_peered":"2024-09-05T13:15:17.613359+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:17.613359+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8372,"log_dups_size":0,"ondisk_log_size":8372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34263040,"num_objects":8365,"num_object_clones":0,"num_object_copies":16730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3914}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8346","reported_seq":8479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:16.974493+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:16.974493+0000","last_peered":"2024-09-05T13:15:16.974493+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:16.974493+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34152448,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8363","reported_seq":14211,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.648634+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:17.648634+0000","last_peered":"2024-09-05T13:15:17.648634+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:17.648634+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8363,"log_dups_size":0,"ondisk_log_size":8363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34234368,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":23832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2868,"num_bytes_recovered":11739136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1519},{"shards":"2","objects":6837}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8235","reported_seq":8353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.649127+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:17.649127+0000","last_peered":"2024-09-05T13:15:17.649127+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:17.649127+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5744,"num_write_kb":22976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3842}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8347","reported_seq":8483,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.070203+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:17.070203+0000","last_peered":"2024-09-05T13:15:17.070203+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:17.070203+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8347,"log_dups_size":0,"ondisk_log_size":8347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34181120,"num_objects":8345,"num_object_clones":0,"num_object_copies":16690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4498,"num_write_kb":17992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3926},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8422","reported_seq":8604,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.090050+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:17.090050+0000","last_peered":"2024-09-05T13:15:17.090050+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:17.090050+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8422,"log_dups_size":0,"ondisk_log_size":8422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4321,"num_write_kb":17284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8474","reported_seq":8608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.013503+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:17.013503+0000","last_peered":"2024-09-05T13:15:17.013503+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:17.013503+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8474,"log_dups_size":0,"ondisk_log_size":8474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34693120,"num_objects":8470,"num_object_clones":0,"num_object_copies":16940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4604,"num_write_kb":18416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3904},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8537","reported_seq":8672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.050877+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:17.050877+0000","last_peered":"2024-09-05T13:15:17.050877+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:17.050877+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8537,"log_dups_size":0,"ondisk_log_size":8537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34918883,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5205,"num_write_kb":20820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3961},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549310947,"num_objects":134110,"num_object_clones":0,"num_object_copies":268220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":134110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89909,"num_write_kb":359636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2871,"num_bytes_recovered":11751424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1106767872,"data_stored":1106684773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134191,"ondisk_log_size":134191,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663304,"kb_used_data":410872,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93708536,"statfs":{"total":96636764160,"available":95957540864,"internally_reserved":0,"allocated":420732928,"data_stored":420487838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784120,"kb_used_data":473448,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93587720,"statfs":{"total":96636764160,"available":95833825280,"internally_reserved":0,"allocated":484810752,"data_stored":484561585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,4,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":372720,"kb_used_data":196640,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93999120,"statfs":{"total":96636764160,"available":96255098880,"internally_reserved":0,"allocated":201359360,"data_stored":201037933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484937728,"data_stored":484934115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":201003008,"data_stored":200927135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420827136,"data_stored":420823523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:20.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:21.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 168 20 135998 135978 3.1608 1.66797 0.102818 0.0247082 2024-09-05T13:15:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:21.388 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:21.424 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-05T13:15:19.566393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549310966,"num_objects":134112,"num_object_clones":0,"num_object_copies":268224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":134112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89911,"num_write_kb":359638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2872,"num_bytes_recovered":11751443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134194,"ondisk_log_size":134194,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1820144,"kb_used_data":1080960,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281295376,"statfs":{"total":289910292480,"available":288046465024,"internally_reserved":0,"allocated":1106903040,"data_stored":1106087356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":756861958},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,6,8,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-412,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5581,"num_write_kb":22324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":1687552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004412"},"pg_stats":[{"pgid":"2.f","version":"43'8302","reported_seq":8372,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.061115+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:17.061115+0000","last_peered":"2024-09-05T13:15:17.061115+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:17.061115+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8302,"log_dups_size":0,"ondisk_log_size":8302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8302,"num_write_kb":33208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3881},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8326","reported_seq":8424,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.623671+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:17.623671+0000","last_peered":"2024-09-05T13:15:17.623671+0000","last_clean":"2024-09-05T13:15:17.623671+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:17.623671+0000","last_undegraded":"2024-09-05T13:15:17.623671+0000","last_fullsized":"2024-09-05T13:15:17.623671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8326,"log_dups_size":0,"ondisk_log_size":8326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34078720,"num_objects":8320,"num_object_clones":0,"num_object_copies":16640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6336,"num_write_kb":25344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8459","reported_seq":8582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.541561+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:17.541561+0000","last_peered":"2024-09-05T13:15:17.541561+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:17.541561+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8459,"log_dups_size":0,"ondisk_log_size":8459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34631680,"num_objects":8455,"num_object_clones":0,"num_object_copies":16910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6270,"num_write_kb":25080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8384","reported_seq":8449,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.624120+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:17.624120+0000","last_peered":"2024-09-05T13:15:17.624120+0000","last_clean":"2024-09-05T13:15:17.624120+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:17.624120+0000","last_undegraded":"2024-09-05T13:15:17.624120+0000","last_fullsized":"2024-09-05T13:15:17.624120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8384,"log_dups_size":0,"ondisk_log_size":8384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":33536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8380","reported_seq":8509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.027577+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:17.027577+0000","last_peered":"2024-09-05T13:15:17.027577+0000","last_clean":"2024-09-05T13:15:17.027577+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:17.027577+0000","last_undegraded":"2024-09-05T13:15:17.027577+0000","last_fullsized":"2024-09-05T13:15:17.027577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4920,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8482","reported_seq":8613,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.046704+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:17.046704+0000","last_peered":"2024-09-05T13:15:17.046704+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:17.046704+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8482,"log_dups_size":0,"ondisk_log_size":8482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4937,"num_write_kb":19748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8293","reported_seq":8428,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:16.966121+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:16.966121+0000","last_peered":"2024-09-05T13:15:16.966121+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:16.966121+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8293,"log_dups_size":0,"ondisk_log_size":8293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33943552,"num_objects":8287,"num_object_clones":0,"num_object_copies":16574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4612,"num_write_kb":18448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3851},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8469","reported_seq":8596,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:16.985146+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:16.985146+0000","last_peered":"2024-09-05T13:15:16.985146+0000","last_clean":"2024-09-05T13:15:16.985146+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:16.985146+0000","last_undegraded":"2024-09-05T13:15:16.985146+0000","last_fullsized":"2024-09-05T13:15:16.985146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8469,"log_dups_size":0,"ondisk_log_size":8469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34680832,"num_objects":8467,"num_object_clones":0,"num_object_copies":16934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5079,"num_write_kb":20316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8372","reported_seq":8502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.613359+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:17.613359+0000","last_peered":"2024-09-05T13:15:17.613359+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:17.613359+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8372,"log_dups_size":0,"ondisk_log_size":8372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34263040,"num_objects":8365,"num_object_clones":0,"num_object_copies":16730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3914}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8346","reported_seq":8479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:16.974493+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:16.974493+0000","last_peered":"2024-09-05T13:15:16.974493+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:16.974493+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34152448,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8363","reported_seq":14211,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.648634+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:17.648634+0000","last_peered":"2024-09-05T13:15:17.648634+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:17.648634+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8363,"log_dups_size":0,"ondisk_log_size":8363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34234368,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":23832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2868,"num_bytes_recovered":11739136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1519},{"shards":"2","objects":6837}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.975670+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:17.975670+0000","last_peered":"2024-09-05T13:13:17.975670+0000","last_clean":"2024-09-05T13:13:17.975670+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:17.975670+0000","last_undegraded":"2024-09-05T13:13:17.975670+0000","last_fullsized":"2024-09-05T13:13:17.975670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8235","reported_seq":8353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.649127+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:17.649127+0000","last_peered":"2024-09-05T13:15:17.649127+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:17.649127+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5744,"num_write_kb":22976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3842}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8347","reported_seq":8483,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.070203+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:17.070203+0000","last_peered":"2024-09-05T13:15:17.070203+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:17.070203+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8347,"log_dups_size":0,"ondisk_log_size":8347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34181120,"num_objects":8345,"num_object_clones":0,"num_object_copies":16690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4498,"num_write_kb":17992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3926},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.105648+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.105648+0000","last_peered":"2024-09-05T13:13:18.105648+0000","last_clean":"2024-09-05T13:13:18.105648+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.105648+0000","last_undegraded":"2024-09-05T13:13:18.105648+0000","last_fullsized":"2024-09-05T13:13:18.105648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8422","reported_seq":8604,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.090050+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:17.090050+0000","last_peered":"2024-09-05T13:15:17.090050+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:17.090050+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8422,"log_dups_size":0,"ondisk_log_size":8422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4321,"num_write_kb":17284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8474","reported_seq":8608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.013503+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:17.013503+0000","last_peered":"2024-09-05T13:15:17.013503+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:17.013503+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8474,"log_dups_size":0,"ondisk_log_size":8474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34693120,"num_objects":8470,"num_object_clones":0,"num_object_copies":16940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4604,"num_write_kb":18416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3904},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.036233+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.036233+0000","last_peered":"2024-09-05T13:13:18.036233+0000","last_clean":"2024-09-05T13:13:18.036233+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.036233+0000","last_undegraded":"2024-09-05T13:13:18.036233+0000","last_fullsized":"2024-09-05T13:13:18.036233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8537","reported_seq":8672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.050877+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:17.050877+0000","last_peered":"2024-09-05T13:15:17.050877+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:17.050877+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8537,"log_dups_size":0,"ondisk_log_size":8537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34918883,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5205,"num_write_kb":20820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3961},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549310947,"num_objects":134110,"num_object_clones":0,"num_object_copies":268220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":134110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89909,"num_write_kb":359636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2871,"num_bytes_recovered":11751424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1106767872,"data_stored":1106684773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134191,"ondisk_log_size":134191,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663304,"kb_used_data":410872,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93708536,"statfs":{"total":96636764160,"available":95957540864,"internally_reserved":0,"allocated":420732928,"data_stored":420487838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784120,"kb_used_data":473448,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93587720,"statfs":{"total":96636764160,"available":95833825280,"internally_reserved":0,"allocated":484810752,"data_stored":484561585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,4,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":372720,"kb_used_data":196640,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93999120,"statfs":{"total":96636764160,"available":96255098880,"internally_reserved":0,"allocated":201359360,"data_stored":201037933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484937728,"data_stored":484934115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":201003008,"data_stored":200927135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420827136,"data_stored":420823523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:21.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:21.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8302", 'reported_seq': 8372, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.061115+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:17.061115+0000', 'last_peered': '2024-09-05T13:15:17.061115+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:17.061115+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 8302, 'log_dups_size': 0, 'ondisk_log_size': 8302, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34004992, 'num_objects': 8302, 'num_object_clones': 0, 'num_object_copies': 16604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8302, 'num_write_kb': 33208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3881}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:21.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:21.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8459", 'reported_seq': 8582, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.541561+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:17.541561+0000', 'last_peered': '2024-09-05T13:15:17.541561+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:17.541561+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 8459, 'log_dups_size': 0, 'ondisk_log_size': 8459, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34631680, 'num_objects': 8455, 'num_object_clones': 0, 'num_object_copies': 16910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6270, 'num_write_kb': 25080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:21.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:21.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8482", 'reported_seq': 8613, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.046704+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:17.046704+0000', 'last_peered': '2024-09-05T13:15:17.046704+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:17.046704+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 8482, 'log_dups_size': 0, 'ondisk_log_size': 8482, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34721792, 'num_objects': 8477, 'num_object_clones': 0, 'num_object_copies': 16954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4937, 'num_write_kb': 19748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4026}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:21.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:21.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8293", 'reported_seq': 8428, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:16.966121+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:16.966121+0000', 'last_peered': '2024-09-05T13:15:16.966121+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:16.966121+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 8293, 'log_dups_size': 0, 'ondisk_log_size': 8293, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33943552, 'num_objects': 8287, 'num_object_clones': 0, 'num_object_copies': 16574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4612, 'num_write_kb': 18448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3851}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:21.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:21.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8372", 'reported_seq': 8502, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.613359+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:17.613359+0000', 'last_peered': '2024-09-05T13:15:17.613359+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:17.613359+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 8372, 'log_dups_size': 0, 'ondisk_log_size': 8372, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34263040, 'num_objects': 8365, 'num_object_clones': 0, 'num_object_copies': 16730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8365, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6026, 'num_write_kb': 24104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 3914}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:21.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:21.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8346", 'reported_seq': 8479, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:16.974493+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:16.974493+0000', 'last_peered': '2024-09-05T13:15:16.974493+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:16.974493+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 8346, 'log_dups_size': 0, 'ondisk_log_size': 8346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34152448, 'num_objects': 8338, 'num_object_clones': 0, 'num_object_copies': 16676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4713, 'num_write_kb': 18852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3884}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:21.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:21.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8363", 'reported_seq': 14211, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.648634+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:17.648634+0000', 'last_peered': '2024-09-05T13:15:17.648634+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:17.648634+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 8363, 'log_dups_size': 0, 'ondisk_log_size': 8363, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34234368, 'num_objects': 8358, 'num_object_clones': 0, 'num_object_copies': 16716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1519, 'num_objects_unfound': 0, 'num_objects_dirty': 8358, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5958, 'num_write_kb': 23832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2868, 'num_bytes_recovered': 11739136, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1519}, {'shards': '2', 'objects': 6837}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:21.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:21.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8235", 'reported_seq': 8353, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.649127+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:17.649127+0000', 'last_peered': '2024-09-05T13:15:17.649127+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:17.649127+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 8235, 'log_dups_size': 0, 'ondisk_log_size': 8235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33705984, 'num_objects': 8229, 'num_object_clones': 0, 'num_object_copies': 16458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5744, 'num_write_kb': 22976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 3842}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:21.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:21.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8347", 'reported_seq': 8483, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.070203+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:17.070203+0000', 'last_peered': '2024-09-05T13:15:17.070203+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:17.070203+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 8347, 'log_dups_size': 0, 'ondisk_log_size': 8347, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34181120, 'num_objects': 8345, 'num_object_clones': 0, 'num_object_copies': 16690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8345, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4498, 'num_write_kb': 17992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3926}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:21.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:21.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8422", 'reported_seq': 8604, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.090050+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:17.090050+0000', 'last_peered': '2024-09-05T13:15:17.090050+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:17.090050+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 8422, 'log_dups_size': 0, 'ondisk_log_size': 8422, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34463744, 'num_objects': 8414, 'num_object_clones': 0, 'num_object_copies': 16828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8414, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4321, 'num_write_kb': 17284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3970}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:21.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:21.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8474", 'reported_seq': 8608, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.013503+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:17.013503+0000', 'last_peered': '2024-09-05T13:15:17.013503+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:17.013503+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 8474, 'log_dups_size': 0, 'ondisk_log_size': 8474, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34693120, 'num_objects': 8470, 'num_object_clones': 0, 'num_object_copies': 16940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8470, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4604, 'num_write_kb': 18416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3904}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:21.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:21.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8537", 'reported_seq': 8672, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:17.050877+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:17.050877+0000', 'last_peered': '2024-09-05T13:15:17.050877+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:17.050877+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 8537, 'log_dups_size': 0, 'ondisk_log_size': 8537, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34918883, 'num_objects': 8526, 'num_object_clones': 0, 'num_object_copies': 17052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5205, 'num_write_kb': 20820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3961}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:21.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:22.163 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/268224 objects degraded (18.309%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":249,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":134112,"data_bytes":549310966,"bytes_used":1878630400,"bytes_avail":288031662080,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":268224,"degraded_ratio":0.18308577905034601,"misplaced_objects":1519,"misplaced_total":268224,"misplaced_ratio":0.0056631770460510607,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93491,"recovering_keys_per_sec":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"write_bytes_sec":1612550,"read_op_per_sec":0,"write_op_per_sec":393},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:22.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:22.186 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:22.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 169 20 136427 136407 3.15201 1.67578 0.034252 0.0247778 2024-09-05T13:15:22.745 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:22.747 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:22.771 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-05T13:15:21.567061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549310966,"num_objects":134112,"num_object_clones":0,"num_object_copies":268224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":134112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89911,"num_write_kb":359638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2872,"num_bytes_recovered":11751443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134194,"ondisk_log_size":134194,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1834600,"kb_used_data":1087096,"kb_used_omap":12,"kb_used_meta":747443,"kb_avail":281280920,"statfs":{"total":289910292480,"available":288031662080,"internally_reserved":0,"allocated":1113186304,"data_stored":1112370620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":765381638},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,4,6,14,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19357696,"num_objects":4726,"num_object_clones":0,"num_object_copies":9452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":4726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4726,"num_write_kb":18904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004395"},"pg_stats":[{"pgid":"2.f","version":"43'8302","reported_seq":8372,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.061115+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:17.061115+0000","last_peered":"2024-09-05T13:15:17.061115+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:17.061115+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8302,"log_dups_size":0,"ondisk_log_size":8302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8302,"num_write_kb":33208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3881},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8326","reported_seq":8424,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.623671+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:17.623671+0000","last_peered":"2024-09-05T13:15:17.623671+0000","last_clean":"2024-09-05T13:15:17.623671+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:17.623671+0000","last_undegraded":"2024-09-05T13:15:17.623671+0000","last_fullsized":"2024-09-05T13:15:17.623671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8326,"log_dups_size":0,"ondisk_log_size":8326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34078720,"num_objects":8320,"num_object_clones":0,"num_object_copies":16640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6336,"num_write_kb":25344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8459","reported_seq":8582,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.541561+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:17.541561+0000","last_peered":"2024-09-05T13:15:17.541561+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:17.541561+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8459,"log_dups_size":0,"ondisk_log_size":8459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34631680,"num_objects":8455,"num_object_clones":0,"num_object_copies":16910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6270,"num_write_kb":25080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8384","reported_seq":8449,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.624120+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:17.624120+0000","last_peered":"2024-09-05T13:15:17.624120+0000","last_clean":"2024-09-05T13:15:17.624120+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:17.624120+0000","last_undegraded":"2024-09-05T13:15:17.624120+0000","last_fullsized":"2024-09-05T13:15:17.624120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8384,"log_dups_size":0,"ondisk_log_size":8384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":33536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8380","reported_seq":8509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:17.027577+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:17.027577+0000","last_peered":"2024-09-05T13:15:17.027577+0000","last_clean":"2024-09-05T13:15:17.027577+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:17.027577+0000","last_undegraded":"2024-09-05T13:15:17.027577+0000","last_fullsized":"2024-09-05T13:15:17.027577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34295808,"num_objects":8373,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4920,"num_write_kb":19680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8482","reported_seq":8613,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.046704+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:17.046704+0000","last_peered":"2024-09-05T13:15:17.046704+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:17.046704+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8482,"log_dups_size":0,"ondisk_log_size":8482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4937,"num_write_kb":19748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8293","reported_seq":8428,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:16.966121+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:16.966121+0000","last_peered":"2024-09-05T13:15:16.966121+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:16.966121+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8293,"log_dups_size":0,"ondisk_log_size":8293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33943552,"num_objects":8287,"num_object_clones":0,"num_object_copies":16574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4612,"num_write_kb":18448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3851},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8469","reported_seq":8596,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:16.985146+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:16.985146+0000","last_peered":"2024-09-05T13:15:16.985146+0000","last_clean":"2024-09-05T13:15:16.985146+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:16.985146+0000","last_undegraded":"2024-09-05T13:15:16.985146+0000","last_fullsized":"2024-09-05T13:15:16.985146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8469,"log_dups_size":0,"ondisk_log_size":8469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34680832,"num_objects":8467,"num_object_clones":0,"num_object_copies":16934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5079,"num_write_kb":20316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8372","reported_seq":8502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.613359+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:17.613359+0000","last_peered":"2024-09-05T13:15:17.613359+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:17.613359+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8372,"log_dups_size":0,"ondisk_log_size":8372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34263040,"num_objects":8365,"num_object_clones":0,"num_object_copies":16730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":3914}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8346","reported_seq":8479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:16.974493+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:16.974493+0000","last_peered":"2024-09-05T13:15:16.974493+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:16.974493+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34152448,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8363","reported_seq":14211,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.648634+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:17.648634+0000","last_peered":"2024-09-05T13:15:17.648634+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:17.648634+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8363,"log_dups_size":0,"ondisk_log_size":8363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34234368,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":23832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2868,"num_bytes_recovered":11739136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1519},{"shards":"2","objects":6837}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8235","reported_seq":8353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.649127+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:17.649127+0000","last_peered":"2024-09-05T13:15:17.649127+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:17.649127+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8235,"log_dups_size":0,"ondisk_log_size":8235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5744,"num_write_kb":22976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3842}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8347","reported_seq":8483,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.070203+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:17.070203+0000","last_peered":"2024-09-05T13:15:17.070203+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:17.070203+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8347,"log_dups_size":0,"ondisk_log_size":8347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34181120,"num_objects":8345,"num_object_clones":0,"num_object_copies":16690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4498,"num_write_kb":17992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3926},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8422","reported_seq":8604,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.090050+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:17.090050+0000","last_peered":"2024-09-05T13:15:17.090050+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:17.090050+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8422,"log_dups_size":0,"ondisk_log_size":8422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4321,"num_write_kb":17284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8474","reported_seq":8608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.013503+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:17.013503+0000","last_peered":"2024-09-05T13:15:17.013503+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:17.013503+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8474,"log_dups_size":0,"ondisk_log_size":8474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34693120,"num_objects":8470,"num_object_clones":0,"num_object_copies":16940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4604,"num_write_kb":18416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3904},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.906165+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.906165+0000","last_peered":"2024-09-05T13:13:17.906165+0000","last_clean":"2024-09-05T13:13:17.906165+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.906165+0000","last_undegraded":"2024-09-05T13:13:17.906165+0000","last_fullsized":"2024-09-05T13:13:17.906165+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8537","reported_seq":8672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:17.050877+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:17.050877+0000","last_peered":"2024-09-05T13:15:17.050877+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:17.050877+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8537,"log_dups_size":0,"ondisk_log_size":8537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34918883,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5205,"num_write_kb":20820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3961},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549310947,"num_objects":134110,"num_object_clones":0,"num_object_copies":268220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1519,"num_objects_unfound":0,"num_objects_dirty":134110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89909,"num_write_kb":359636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2871,"num_bytes_recovered":11751424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1114664960,"data_stored":1114581861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134191,"ondisk_log_size":134191,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663304,"kb_used_data":410872,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93708536,"statfs":{"total":96636764160,"available":95957540864,"internally_reserved":0,"allocated":420732928,"data_stored":420487838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784120,"kb_used_data":473448,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93587720,"statfs":{"total":96636764160,"available":95833825280,"internally_reserved":0,"allocated":484810752,"data_stored":484561585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,4,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":387176,"kb_used_data":202776,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93984664,"statfs":{"total":96636764160,"available":96240295936,"internally_reserved":0,"allocated":207642624,"data_stored":207321197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":188804783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484937728,"data_stored":484934115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208900096,"data_stored":208824223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420827136,"data_stored":420823523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:23.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 170 20 136888 136868 3.14406 1.80078 0.035049 0.02484 2024-09-05T13:15:24.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 171 20 137380 137360 3.13692 1.92188 0.0193481 0.0248963 2024-09-05T13:15:25.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 172 20 137877 137857 3.12997 1.94141 0.0738757 0.0249524 2024-09-05T13:15:25.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:26.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 173 20 138338 138318 3.12228 1.80078 0.0425893 0.0250143 2024-09-05T13:15:26.326 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:26.327 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:26.350 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-05T13:15:25.568710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558940662,"num_objects":136463,"num_object_clones":0,"num_object_copies":272926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":136463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92262,"num_write_kb":369042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3010,"num_bytes_recovered":12316691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136545,"ondisk_log_size":136545,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1847064,"kb_used_data":1106472,"kb_used_omap":12,"kb_used_meta":740531,"kb_avail":281268456,"statfs":{"total":289910292480,"available":288018898944,"internally_reserved":0,"allocated":1133027328,"data_stored":1132211644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":758303750},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,6,9,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19308544,"num_objects":4714,"num_object_clones":0,"num_object_copies":9428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":4714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4714,"num_write_kb":18856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004601"},"pg_stats":[{"pgid":"2.f","version":"43'8431","reported_seq":8501,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.083155+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:22.083155+0000","last_peered":"2024-09-05T13:15:22.083155+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:22.083155+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8431,"log_dups_size":0,"ondisk_log_size":8431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533376,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8431,"num_write_kb":33724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4010},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8449","reported_seq":8548,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.609138+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:22.609138+0000","last_peered":"2024-09-05T13:15:22.609138+0000","last_clean":"2024-09-05T13:15:22.609138+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:22.609138+0000","last_undegraded":"2024-09-05T13:15:22.609138+0000","last_fullsized":"2024-09-05T13:15:22.609138+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8449,"log_dups_size":0,"ondisk_log_size":8449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34582528,"num_objects":8443,"num_object_clones":0,"num_object_copies":16886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6459,"num_write_kb":25836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8622","reported_seq":8745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.550571+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:22.550571+0000","last_peered":"2024-09-05T13:15:22.550571+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:22.550571+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8622,"log_dups_size":0,"ondisk_log_size":8622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8528","reported_seq":8593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.539907+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:22.539907+0000","last_peered":"2024-09-05T13:15:22.539907+0000","last_clean":"2024-09-05T13:15:22.539907+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:22.539907+0000","last_undegraded":"2024-09-05T13:15:22.539907+0000","last_fullsized":"2024-09-05T13:15:22.539907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8528,"log_dups_size":0,"ondisk_log_size":8528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8528,"num_write_kb":34112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8520","reported_seq":8650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.021932+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:22.021932+0000","last_peered":"2024-09-05T13:15:22.021932+0000","last_clean":"2024-09-05T13:15:22.021932+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:22.021932+0000","last_undegraded":"2024-09-05T13:15:22.021932+0000","last_fullsized":"2024-09-05T13:15:22.021932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34869248,"num_objects":8513,"num_object_clones":0,"num_object_copies":17026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5060,"num_write_kb":20240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8631","reported_seq":8764,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:21.921624+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:21.921624+0000","last_peered":"2024-09-05T13:15:21.921624+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:21.921624+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332096,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5086,"num_write_kb":20344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4175},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8435","reported_seq":8570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.044586+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:22.044586+0000","last_peered":"2024-09-05T13:15:22.044586+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:22.044586+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8435,"log_dups_size":0,"ondisk_log_size":8435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34525184,"num_objects":8429,"num_object_clones":0,"num_object_copies":16858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3993},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8611","reported_seq":8738,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.067447+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:22.067447+0000","last_peered":"2024-09-05T13:15:22.067447+0000","last_clean":"2024-09-05T13:15:22.067447+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:22.067447+0000","last_undegraded":"2024-09-05T13:15:22.067447+0000","last_fullsized":"2024-09-05T13:15:22.067447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8611,"log_dups_size":0,"ondisk_log_size":8611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5221,"num_write_kb":20884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8527","reported_seq":8657,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.624680+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:22.624680+0000","last_peered":"2024-09-05T13:15:22.624680+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:22.624680+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8527,"log_dups_size":0,"ondisk_log_size":8527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34897920,"num_objects":8520,"num_object_clones":0,"num_object_copies":17040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6181,"num_write_kb":24724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4069}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8511","reported_seq":8643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.053068+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:22.053068+0000","last_peered":"2024-09-05T13:15:22.053068+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:22.053068+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8511,"log_dups_size":0,"ondisk_log_size":8511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4878,"num_write_kb":19512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4049},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8519","reported_seq":14642,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.656631+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:22.656631+0000","last_peered":"2024-09-05T13:15:22.656631+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:22.656631+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8519,"log_dups_size":0,"ondisk_log_size":8519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6114,"num_write_kb":24456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3006,"num_bytes_recovered":12304384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1381},{"shards":"2","objects":7131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8374","reported_seq":8492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.589785+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:22.589785+0000","last_peered":"2024-09-05T13:15:22.589785+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:22.589785+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275328,"num_objects":8368,"num_object_clones":0,"num_object_copies":16736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5883,"num_write_kb":23532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3981}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8507","reported_seq":8643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.014069+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:22.014069+0000","last_peered":"2024-09-05T13:15:22.014069+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:22.014069+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8507,"log_dups_size":0,"ondisk_log_size":8507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34836480,"num_objects":8505,"num_object_clones":0,"num_object_copies":17010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":18632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4086},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8574","reported_seq":8757,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.014073+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:22.014073+0000","last_peered":"2024-09-05T13:15:22.014073+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:22.014073+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35086336,"num_objects":8566,"num_object_clones":0,"num_object_copies":17132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4473,"num_write_kb":17892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4122},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8606","reported_seq":8739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.049564+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:22.049564+0000","last_peered":"2024-09-05T13:15:22.049564+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:22.049564+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8606,"log_dups_size":0,"ondisk_log_size":8606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35233792,"num_objects":8602,"num_object_clones":0,"num_object_copies":17204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4736,"num_write_kb":18944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4036},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8697","reported_seq":8832,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.007790+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:22.007790+0000","last_peered":"2024-09-05T13:15:22.007790+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:22.007790+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8697,"log_dups_size":0,"ondisk_log_size":8697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35574243,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5365,"num_write_kb":21460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4121},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558940643,"num_objects":136461,"num_object_clones":0,"num_object_copies":272922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":136461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92260,"num_write_kb":369040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3009,"num_bytes_recovered":12316672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1134444544,"data_stored":1134361445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136542,"ondisk_log_size":136542,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667948,"kb_used_data":415516,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93703892,"statfs":{"total":96636764160,"available":95952785408,"internally_reserved":0,"allocated":425488384,"data_stored":425243294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":778608,"kb_used_data":480416,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93593232,"statfs":{"total":96636764160,"available":95839469568,"internally_reserved":0,"allocated":491945984,"data_stored":491696817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400508,"kb_used_data":210540,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93971332,"statfs":{"total":96636764160,"available":96226643968,"internally_reserved":0,"allocated":215592960,"data_stored":215271533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":194506415},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492097536,"data_stored":492093923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":216764416,"data_stored":216688543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425582592,"data_stored":425578979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:26.352 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:26.922 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:26.924 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:26.949 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-05T13:15:25.568710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558940662,"num_objects":136463,"num_object_clones":0,"num_object_copies":272926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":136463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92262,"num_write_kb":369042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3010,"num_bytes_recovered":12316691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136545,"ondisk_log_size":136545,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1847064,"kb_used_data":1106472,"kb_used_omap":12,"kb_used_meta":740531,"kb_avail":281268456,"statfs":{"total":289910292480,"available":288018898944,"internally_reserved":0,"allocated":1133027328,"data_stored":1132211644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":758303750},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,6,9,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19308544,"num_objects":4714,"num_object_clones":0,"num_object_copies":9428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":4714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4714,"num_write_kb":18856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004601"},"pg_stats":[{"pgid":"2.f","version":"43'8431","reported_seq":8501,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.083155+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:22.083155+0000","last_peered":"2024-09-05T13:15:22.083155+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:22.083155+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8431,"log_dups_size":0,"ondisk_log_size":8431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34533376,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8431,"num_write_kb":33724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4010},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8449","reported_seq":8548,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.609138+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:22.609138+0000","last_peered":"2024-09-05T13:15:22.609138+0000","last_clean":"2024-09-05T13:15:22.609138+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:22.609138+0000","last_undegraded":"2024-09-05T13:15:22.609138+0000","last_fullsized":"2024-09-05T13:15:22.609138+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8449,"log_dups_size":0,"ondisk_log_size":8449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34582528,"num_objects":8443,"num_object_clones":0,"num_object_copies":16886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6459,"num_write_kb":25836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8622","reported_seq":8745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.550571+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:22.550571+0000","last_peered":"2024-09-05T13:15:22.550571+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:22.550571+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8622,"log_dups_size":0,"ondisk_log_size":8622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8528","reported_seq":8593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.539907+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:22.539907+0000","last_peered":"2024-09-05T13:15:22.539907+0000","last_clean":"2024-09-05T13:15:22.539907+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:22.539907+0000","last_undegraded":"2024-09-05T13:15:22.539907+0000","last_fullsized":"2024-09-05T13:15:22.539907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8528,"log_dups_size":0,"ondisk_log_size":8528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8528,"num_write_kb":34112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8520","reported_seq":8650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.021932+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:22.021932+0000","last_peered":"2024-09-05T13:15:22.021932+0000","last_clean":"2024-09-05T13:15:22.021932+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:22.021932+0000","last_undegraded":"2024-09-05T13:15:22.021932+0000","last_fullsized":"2024-09-05T13:15:22.021932+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8520,"log_dups_size":0,"ondisk_log_size":8520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34869248,"num_objects":8513,"num_object_clones":0,"num_object_copies":17026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5060,"num_write_kb":20240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8631","reported_seq":8764,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:21.921624+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:21.921624+0000","last_peered":"2024-09-05T13:15:21.921624+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:21.921624+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332096,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5086,"num_write_kb":20344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4175},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8435","reported_seq":8570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.044586+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:22.044586+0000","last_peered":"2024-09-05T13:15:22.044586+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:22.044586+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8435,"log_dups_size":0,"ondisk_log_size":8435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34525184,"num_objects":8429,"num_object_clones":0,"num_object_copies":16858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3993},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8611","reported_seq":8738,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.067447+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:22.067447+0000","last_peered":"2024-09-05T13:15:22.067447+0000","last_clean":"2024-09-05T13:15:22.067447+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:22.067447+0000","last_undegraded":"2024-09-05T13:15:22.067447+0000","last_fullsized":"2024-09-05T13:15:22.067447+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8611,"log_dups_size":0,"ondisk_log_size":8611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5221,"num_write_kb":20884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8527","reported_seq":8657,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.624680+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:22.624680+0000","last_peered":"2024-09-05T13:15:22.624680+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:22.624680+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8527,"log_dups_size":0,"ondisk_log_size":8527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34897920,"num_objects":8520,"num_object_clones":0,"num_object_copies":17040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6181,"num_write_kb":24724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4069}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8511","reported_seq":8643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.053068+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:22.053068+0000","last_peered":"2024-09-05T13:15:22.053068+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:22.053068+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8511,"log_dups_size":0,"ondisk_log_size":8511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828288,"num_objects":8503,"num_object_clones":0,"num_object_copies":17006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4878,"num_write_kb":19512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4049},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8519","reported_seq":14642,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.656631+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:22.656631+0000","last_peered":"2024-09-05T13:15:22.656631+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:22.656631+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8519,"log_dups_size":0,"ondisk_log_size":8519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6114,"num_write_kb":24456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3006,"num_bytes_recovered":12304384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1381},{"shards":"2","objects":7131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8374","reported_seq":8492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.589785+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:22.589785+0000","last_peered":"2024-09-05T13:15:22.589785+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:22.589785+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275328,"num_objects":8368,"num_object_clones":0,"num_object_copies":16736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5883,"num_write_kb":23532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3981}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8507","reported_seq":8643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.014069+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:22.014069+0000","last_peered":"2024-09-05T13:15:22.014069+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:22.014069+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8507,"log_dups_size":0,"ondisk_log_size":8507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34836480,"num_objects":8505,"num_object_clones":0,"num_object_copies":17010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":18632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4086},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8574","reported_seq":8757,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.014073+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:22.014073+0000","last_peered":"2024-09-05T13:15:22.014073+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:22.014073+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35086336,"num_objects":8566,"num_object_clones":0,"num_object_copies":17132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4473,"num_write_kb":17892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4122},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8606","reported_seq":8739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.049564+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:22.049564+0000","last_peered":"2024-09-05T13:15:22.049564+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:22.049564+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8606,"log_dups_size":0,"ondisk_log_size":8606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35233792,"num_objects":8602,"num_object_clones":0,"num_object_copies":17204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4736,"num_write_kb":18944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4036},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8697","reported_seq":8832,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.007790+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:22.007790+0000","last_peered":"2024-09-05T13:15:22.007790+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:22.007790+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8697,"log_dups_size":0,"ondisk_log_size":8697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35574243,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5365,"num_write_kb":21460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4121},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558940643,"num_objects":136461,"num_object_clones":0,"num_object_copies":272922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":136461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92260,"num_write_kb":369040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3009,"num_bytes_recovered":12316672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1134444544,"data_stored":1134361445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136542,"ondisk_log_size":136542,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667948,"kb_used_data":415516,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93703892,"statfs":{"total":96636764160,"available":95952785408,"internally_reserved":0,"allocated":425488384,"data_stored":425243294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":778608,"kb_used_data":480416,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93593232,"statfs":{"total":96636764160,"available":95839469568,"internally_reserved":0,"allocated":491945984,"data_stored":491696817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400508,"kb_used_data":210540,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93971332,"statfs":{"total":96636764160,"available":96226643968,"internally_reserved":0,"allocated":215592960,"data_stored":215271533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":194506415},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492097536,"data_stored":492093923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":216764416,"data_stored":216688543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425582592,"data_stored":425578979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:26.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:26.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8431", 'reported_seq': 8501, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.083155+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:22.083155+0000', 'last_peered': '2024-09-05T13:15:22.083155+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:22.083155+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 8431, 'log_dups_size': 0, 'ondisk_log_size': 8431, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34533376, 'num_objects': 8431, 'num_object_clones': 0, 'num_object_copies': 16862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8431, 'num_write_kb': 33724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4010}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:26.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:26.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8622", 'reported_seq': 8745, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.550571+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:22.550571+0000', 'last_peered': '2024-09-05T13:15:22.550571+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:22.550571+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 8622, 'log_dups_size': 0, 'ondisk_log_size': 8622, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35299328, 'num_objects': 8618, 'num_object_clones': 0, 'num_object_copies': 17236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8618, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6433, 'num_write_kb': 25732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 4126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8631", 'reported_seq': 8764, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:21.921624+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:21.921624+0000', 'last_peered': '2024-09-05T13:15:21.921624+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:21.921624+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 8631, 'log_dups_size': 0, 'ondisk_log_size': 8631, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35332096, 'num_objects': 8626, 'num_object_clones': 0, 'num_object_copies': 17252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5086, 'num_write_kb': 20344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4175}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8435", 'reported_seq': 8570, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.044586+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:22.044586+0000', 'last_peered': '2024-09-05T13:15:22.044586+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:22.044586+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 8435, 'log_dups_size': 0, 'ondisk_log_size': 8435, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34525184, 'num_objects': 8429, 'num_object_clones': 0, 'num_object_copies': 16858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4754, 'num_write_kb': 19016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3993}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8527", 'reported_seq': 8657, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.624680+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:22.624680+0000', 'last_peered': '2024-09-05T13:15:22.624680+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:22.624680+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 8527, 'log_dups_size': 0, 'ondisk_log_size': 8527, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34897920, 'num_objects': 8520, 'num_object_clones': 0, 'num_object_copies': 17040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6181, 'num_write_kb': 24724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 4069}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8511", 'reported_seq': 8643, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.053068+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:22.053068+0000', 'last_peered': '2024-09-05T13:15:22.053068+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:22.053068+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 8511, 'log_dups_size': 0, 'ondisk_log_size': 8511, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34828288, 'num_objects': 8503, 'num_object_clones': 0, 'num_object_copies': 17006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4878, 'num_write_kb': 19512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4049}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:26.951 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8519", 'reported_seq': 14642, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.656631+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:22.656631+0000', 'last_peered': '2024-09-05T13:15:22.656631+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:22.656631+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 8519, 'log_dups_size': 0, 'ondisk_log_size': 8519, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34873344, 'num_objects': 8514, 'num_object_clones': 0, 'num_object_copies': 17028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1381, 'num_objects_unfound': 0, 'num_objects_dirty': 8514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6114, 'num_write_kb': 24456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3006, 'num_bytes_recovered': 12304384, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1381}, {'shards': '2', 'objects': 7131}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8374", 'reported_seq': 8492, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.589785+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:22.589785+0000', 'last_peered': '2024-09-05T13:15:22.589785+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:22.589785+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 8374, 'log_dups_size': 0, 'ondisk_log_size': 8374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34275328, 'num_objects': 8368, 'num_object_clones': 0, 'num_object_copies': 16736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8368, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5883, 'num_write_kb': 23532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 3981}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8507", 'reported_seq': 8643, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.014069+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:22.014069+0000', 'last_peered': '2024-09-05T13:15:22.014069+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:22.014069+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 8507, 'log_dups_size': 0, 'ondisk_log_size': 8507, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34836480, 'num_objects': 8505, 'num_object_clones': 0, 'num_object_copies': 17010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4658, 'num_write_kb': 18632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4086}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8574", 'reported_seq': 8757, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.014073+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:22.014073+0000', 'last_peered': '2024-09-05T13:15:22.014073+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:22.014073+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 8574, 'log_dups_size': 0, 'ondisk_log_size': 8574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35086336, 'num_objects': 8566, 'num_object_clones': 0, 'num_object_copies': 17132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4473, 'num_write_kb': 17892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4122}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8606", 'reported_seq': 8739, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.049564+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:22.049564+0000', 'last_peered': '2024-09-05T13:15:22.049564+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:22.049564+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 8606, 'log_dups_size': 0, 'ondisk_log_size': 8606, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35233792, 'num_objects': 8602, 'num_object_clones': 0, 'num_object_copies': 17204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8602, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4736, 'num_write_kb': 18944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4036}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:26.952 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8697", 'reported_seq': 8832, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:22.007790+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:22.007790+0000', 'last_peered': '2024-09-05T13:15:22.007790+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:22.007790+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 8697, 'log_dups_size': 0, 'ondisk_log_size': 8697, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35574243, 'num_objects': 8686, 'num_object_clones': 0, 'num_object_copies': 17372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5365, 'num_write_kb': 21460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4121}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:26.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:27.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 174 20 138776 138756 3.11417 1.71094 0.053407 0.0250801 2024-09-05T13:15:27.658 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:27.658 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/272926 objects degraded (17.993%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":254,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":136463,"data_bytes":558940662,"bytes_used":1891393536,"bytes_avail":288018898944,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":272926,"degraded_ratio":0.17993155653913515,"misplaced_objects":1381,"misplaced_total":272926,"misplaced_ratio":0.0050599796281776011,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96901,"recovering_keys_per_sec":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"write_bytes_sec":1608428,"read_op_per_sec":0,"write_op_per_sec":392},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:27.691 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:27.692 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:28.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 175 20 139197 139177 3.10578 1.64453 0.101666 0.0251474 2024-09-05T13:15:28.253 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:28.253 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:28.285 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-05T13:15:27.569376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565047798,"num_objects":137954,"num_object_clones":0,"num_object_copies":275908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":137954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93753,"num_write_kb":375006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3010,"num_bytes_recovered":12316691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138036,"ondisk_log_size":138036,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1858408,"kb_used_data":1113656,"kb_used_omap":12,"kb_used_meta":744691,"kb_avail":281257112,"statfs":{"total":289910292480,"available":288007282688,"internally_reserved":0,"allocated":1140383744,"data_stored":1139568060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":762563590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,8,13,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25415680,"num_objects":6205,"num_object_clones":0,"num_object_copies":12410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":6205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6205,"num_write_kb":24820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004462"},"pg_stats":[{"pgid":"2.f","version":"43'8573","reported_seq":8643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.100903+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:27.100903+0000","last_peered":"2024-09-05T13:15:27.100903+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:27.100903+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8573,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4152},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8449","reported_seq":8548,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.609138+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:22.609138+0000","last_peered":"2024-09-05T13:15:22.609138+0000","last_clean":"2024-09-05T13:15:22.609138+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:22.609138+0000","last_undegraded":"2024-09-05T13:15:22.609138+0000","last_fullsized":"2024-09-05T13:15:22.609138+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8449,"log_dups_size":0,"ondisk_log_size":8449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34582528,"num_objects":8443,"num_object_clones":0,"num_object_copies":16886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6459,"num_write_kb":25836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8622","reported_seq":8745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.550571+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:22.550571+0000","last_peered":"2024-09-05T13:15:22.550571+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:22.550571+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8622,"log_dups_size":0,"ondisk_log_size":8622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6433,"num_write_kb":25732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8528","reported_seq":8593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:22.539907+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:22.539907+0000","last_peered":"2024-09-05T13:15:22.539907+0000","last_clean":"2024-09-05T13:15:22.539907+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:22.539907+0000","last_undegraded":"2024-09-05T13:15:22.539907+0000","last_fullsized":"2024-09-05T13:15:22.539907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8528,"log_dups_size":0,"ondisk_log_size":8528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8528,"num_write_kb":34112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8663","reported_seq":8793,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:26.981863+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:26.981863+0000","last_peered":"2024-09-05T13:15:26.981863+0000","last_clean":"2024-09-05T13:15:26.981863+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:26.981863+0000","last_undegraded":"2024-09-05T13:15:26.981863+0000","last_fullsized":"2024-09-05T13:15:26.981863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35454976,"num_objects":8656,"num_object_clones":0,"num_object_copies":17312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5203,"num_write_kb":20812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8793","reported_seq":8927,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.105336+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:27.105336+0000","last_peered":"2024-09-05T13:15:27.105336+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:27.105336+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8793,"log_dups_size":0,"ondisk_log_size":8793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4337},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8569","reported_seq":8704,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.119151+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:27.119151+0000","last_peered":"2024-09-05T13:15:27.119151+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:27.119151+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8569,"log_dups_size":0,"ondisk_log_size":8569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074048,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4127},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8768","reported_seq":8895,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:27.052093+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:27.052093+0000","last_peered":"2024-09-05T13:15:27.052093+0000","last_clean":"2024-09-05T13:15:27.052093+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:27.052093+0000","last_undegraded":"2024-09-05T13:15:27.052093+0000","last_fullsized":"2024-09-05T13:15:27.052093+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35905536,"num_objects":8766,"num_object_clones":0,"num_object_copies":17532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5378,"num_write_kb":21512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8527","reported_seq":8657,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.624680+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:22.624680+0000","last_peered":"2024-09-05T13:15:22.624680+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:22.624680+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8527,"log_dups_size":0,"ondisk_log_size":8527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34897920,"num_objects":8520,"num_object_clones":0,"num_object_copies":17040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6181,"num_write_kb":24724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4069}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8669","reported_seq":8802,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.142773+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:27.142773+0000","last_peered":"2024-09-05T13:15:27.142773+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:27.142773+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8669,"log_dups_size":0,"ondisk_log_size":8669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35475456,"num_objects":8661,"num_object_clones":0,"num_object_copies":17322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5036,"num_write_kb":20144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4207},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8519","reported_seq":14642,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.656631+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:22.656631+0000","last_peered":"2024-09-05T13:15:22.656631+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:22.656631+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8519,"log_dups_size":0,"ondisk_log_size":8519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6114,"num_write_kb":24456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3006,"num_bytes_recovered":12304384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1381},{"shards":"2","objects":7131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8374","reported_seq":8492,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:22.589785+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:22.589785+0000","last_peered":"2024-09-05T13:15:22.589785+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:22.589785+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275328,"num_objects":8368,"num_object_clones":0,"num_object_copies":16736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5883,"num_write_kb":23532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":3981}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8657","reported_seq":8792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.103869+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:27.103869+0000","last_peered":"2024-09-05T13:15:27.103869+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:27.103869+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4236},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8722","reported_seq":8905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:26.967027+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:26.967027+0000","last_peered":"2024-09-05T13:15:26.967027+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:26.967027+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8722,"log_dups_size":0,"ondisk_log_size":8722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35692544,"num_objects":8714,"num_object_clones":0,"num_object_copies":17428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4621,"num_write_kb":18484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4270},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8757","reported_seq":8891,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.099615+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:27.099615+0000","last_peered":"2024-09-05T13:15:27.099615+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:27.099615+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8757,"log_dups_size":0,"ondisk_log_size":8757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4887,"num_write_kb":19548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4187},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8843","reported_seq":8976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.057652+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:27.057652+0000","last_peered":"2024-09-05T13:15:27.057652+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:27.057652+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8843,"log_dups_size":0,"ondisk_log_size":8843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36172259,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5511,"num_write_kb":22044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4267},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565047779,"num_objects":137952,"num_object_clones":0,"num_object_copies":275904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":137952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93751,"num_write_kb":375004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3009,"num_bytes_recovered":12316672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1141706752,"data_stored":1141623653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138033,"ondisk_log_size":138033,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667948,"kb_used_data":415516,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93703892,"statfs":{"total":96636764160,"available":95952785408,"internally_reserved":0,"allocated":425488384,"data_stored":425243294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789952,"kb_used_data":487600,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93581888,"statfs":{"total":96636764160,"available":95827853312,"internally_reserved":0,"allocated":499302400,"data_stored":499053233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400508,"kb_used_data":210540,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93971332,"statfs":{"total":96636764160,"available":96226643968,"internally_reserved":0,"allocated":215592960,"data_stored":215271533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":194506415},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499359744,"data_stored":499356131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":216764416,"data_stored":216688543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425582592,"data_stored":425578979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:29.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 176 20 139658 139638 3.09836 1.80078 0.0667845 0.0252092 2024-09-05T13:15:30.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 177 20 140151 140131 3.09174 1.92578 0.0201699 0.025261 2024-09-05T13:15:31.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 178 20 140639 140619 3.08507 1.90625 0.114459 0.0253168 2024-09-05T13:15:31.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:31.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:31.872 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-05T13:15:31.570863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568517110,"num_objects":138801,"num_object_clones":0,"num_object_copies":277602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1232,"num_objects_unfound":0,"num_objects_dirty":138801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94600,"num_write_kb":378394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3159,"num_bytes_recovered":12926995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138883,"ondisk_log_size":138883,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1883108,"kb_used_data":1125876,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281232412,"statfs":{"total":289910292480,"available":287981989888,"internally_reserved":0,"allocated":1152897024,"data_stored":1152081340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":775343110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,7,8,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19206144,"num_objects":4689,"num_object_clones":0,"num_object_copies":9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4689,"num_write_kb":18756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004470"},"pg_stats":[{"pgid":"2.f","version":"43'8573","reported_seq":8643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.100903+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:27.100903+0000","last_peered":"2024-09-05T13:15:27.100903+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:27.100903+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8573,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4152},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8585","reported_seq":8685,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:27.500001+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:27.500001+0000","last_peered":"2024-09-05T13:15:27.500001+0000","last_clean":"2024-09-05T13:15:27.500001+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:27.500001+0000","last_undegraded":"2024-09-05T13:15:27.500001+0000","last_fullsized":"2024-09-05T13:15:27.500001+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8585,"log_dups_size":0,"ondisk_log_size":8585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35139584,"num_objects":8579,"num_object_clones":0,"num_object_copies":17158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6595,"num_write_kb":26380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8768","reported_seq":8891,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.627269+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:27.627269+0000","last_peered":"2024-09-05T13:15:27.627269+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:27.627269+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35897344,"num_objects":8764,"num_object_clones":0,"num_object_copies":17528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4272}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8674","reported_seq":8739,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:27.624700+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:27.624700+0000","last_peered":"2024-09-05T13:15:27.624700+0000","last_clean":"2024-09-05T13:15:27.624700+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:27.624700+0000","last_undegraded":"2024-09-05T13:15:27.624700+0000","last_fullsized":"2024-09-05T13:15:27.624700+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8674,"log_dups_size":0,"ondisk_log_size":8674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35528704,"num_objects":8674,"num_object_clones":0,"num_object_copies":17348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8674,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8663","reported_seq":8793,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:26.981863+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:26.981863+0000","last_peered":"2024-09-05T13:15:26.981863+0000","last_clean":"2024-09-05T13:15:26.981863+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:26.981863+0000","last_undegraded":"2024-09-05T13:15:26.981863+0000","last_fullsized":"2024-09-05T13:15:26.981863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35454976,"num_objects":8656,"num_object_clones":0,"num_object_copies":17312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5203,"num_write_kb":20812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8793","reported_seq":8927,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.105336+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:27.105336+0000","last_peered":"2024-09-05T13:15:27.105336+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:27.105336+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8793,"log_dups_size":0,"ondisk_log_size":8793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4337},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8569","reported_seq":8704,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.119151+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:27.119151+0000","last_peered":"2024-09-05T13:15:27.119151+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:27.119151+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8569,"log_dups_size":0,"ondisk_log_size":8569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074048,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4127},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8768","reported_seq":8895,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:27.052093+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:27.052093+0000","last_peered":"2024-09-05T13:15:27.052093+0000","last_clean":"2024-09-05T13:15:27.052093+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:27.052093+0000","last_undegraded":"2024-09-05T13:15:27.052093+0000","last_fullsized":"2024-09-05T13:15:27.052093+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35905536,"num_objects":8766,"num_object_clones":0,"num_object_copies":17532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5378,"num_write_kb":21512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8663","reported_seq":8793,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.589582+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:27.589582+0000","last_peered":"2024-09-05T13:15:27.589582+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:27.589582+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35454976,"num_objects":8656,"num_object_clones":0,"num_object_copies":17312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6317,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8669","reported_seq":8802,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.142773+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:27.142773+0000","last_peered":"2024-09-05T13:15:27.142773+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:27.142773+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8669,"log_dups_size":0,"ondisk_log_size":8669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35475456,"num_objects":8661,"num_object_clones":0,"num_object_copies":17322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5036,"num_write_kb":20144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4207},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8676","reported_seq":15097,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.664382+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:27.664382+0000","last_peered":"2024-09-05T13:15:27.664382+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:27.664382+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8676,"log_dups_size":0,"ondisk_log_size":8676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516416,"num_objects":8671,"num_object_clones":0,"num_object_copies":17342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1232,"num_objects_unfound":0,"num_objects_dirty":8671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6271,"num_write_kb":25084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3155,"num_bytes_recovered":12914688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1232},{"shards":"2","objects":7437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8500","reported_seq":8618,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.564903+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:27.564903+0000","last_peered":"2024-09-05T13:15:27.564903+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:27.564903+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8500,"log_dups_size":0,"ondisk_log_size":8500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34791424,"num_objects":8494,"num_object_clones":0,"num_object_copies":16988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":24036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4107}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8657","reported_seq":8792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.103869+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:27.103869+0000","last_peered":"2024-09-05T13:15:27.103869+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:27.103869+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4236},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8722","reported_seq":8905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:26.967027+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:26.967027+0000","last_peered":"2024-09-05T13:15:26.967027+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:26.967027+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8722,"log_dups_size":0,"ondisk_log_size":8722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35692544,"num_objects":8714,"num_object_clones":0,"num_object_copies":17428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4621,"num_write_kb":18484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4270},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8757","reported_seq":8891,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.099615+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:27.099615+0000","last_peered":"2024-09-05T13:15:27.099615+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:27.099615+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8757,"log_dups_size":0,"ondisk_log_size":8757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4887,"num_write_kb":19548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4187},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8843","reported_seq":8976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.057652+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:27.057652+0000","last_peered":"2024-09-05T13:15:27.057652+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:27.057652+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8843,"log_dups_size":0,"ondisk_log_size":8843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36172259,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5511,"num_write_kb":22044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4267},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568517091,"num_objects":138799,"num_object_clones":0,"num_object_copies":277598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1232,"num_objects_unfound":0,"num_objects_dirty":138799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94598,"num_write_kb":378392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3158,"num_bytes_recovered":12926976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1154166784,"data_stored":1154083685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138880,"ondisk_log_size":138880,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676752,"kb_used_data":420160,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93695088,"statfs":{"total":96636764160,"available":95943770112,"internally_reserved":0,"allocated":430243840,"data_stored":429998750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789952,"kb_used_data":487600,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93581888,"statfs":{"total":96636764160,"available":95827853312,"internally_reserved":0,"allocated":499302400,"data_stored":499053233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416404,"kb_used_data":218116,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93955436,"statfs":{"total":96636764160,"available":96210366464,"internally_reserved":0,"allocated":223350784,"data_stored":223029357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499359744,"data_stored":499356131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224555008,"data_stored":224479135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430252032,"data_stored":430248419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:31.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:32.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 179 20 141071 141051 3.07726 1.6875 0.0328005 0.0253788 2024-09-05T13:15:32.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:32.426 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:32.458 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-05T13:15:31.570863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568517110,"num_objects":138801,"num_object_clones":0,"num_object_copies":277602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1232,"num_objects_unfound":0,"num_objects_dirty":138801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94600,"num_write_kb":378394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3159,"num_bytes_recovered":12926995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138883,"ondisk_log_size":138883,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1883108,"kb_used_data":1125876,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281232412,"statfs":{"total":289910292480,"available":287981989888,"internally_reserved":0,"allocated":1152897024,"data_stored":1152081340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":775343110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,7,8,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19206144,"num_objects":4689,"num_object_clones":0,"num_object_copies":9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4689,"num_write_kb":18756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004470"},"pg_stats":[{"pgid":"2.f","version":"43'8573","reported_seq":8643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.100903+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:27.100903+0000","last_peered":"2024-09-05T13:15:27.100903+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:27.100903+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8573,"log_dups_size":0,"ondisk_log_size":8573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8573,"num_write_kb":34292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4152},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8585","reported_seq":8685,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:27.500001+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:27.500001+0000","last_peered":"2024-09-05T13:15:27.500001+0000","last_clean":"2024-09-05T13:15:27.500001+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:27.500001+0000","last_undegraded":"2024-09-05T13:15:27.500001+0000","last_fullsized":"2024-09-05T13:15:27.500001+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8585,"log_dups_size":0,"ondisk_log_size":8585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35139584,"num_objects":8579,"num_object_clones":0,"num_object_copies":17158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6595,"num_write_kb":26380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8768","reported_seq":8891,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.627269+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:27.627269+0000","last_peered":"2024-09-05T13:15:27.627269+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:27.627269+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35897344,"num_objects":8764,"num_object_clones":0,"num_object_copies":17528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4272}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8674","reported_seq":8739,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:27.624700+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:27.624700+0000","last_peered":"2024-09-05T13:15:27.624700+0000","last_clean":"2024-09-05T13:15:27.624700+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:27.624700+0000","last_undegraded":"2024-09-05T13:15:27.624700+0000","last_fullsized":"2024-09-05T13:15:27.624700+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8674,"log_dups_size":0,"ondisk_log_size":8674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35528704,"num_objects":8674,"num_object_clones":0,"num_object_copies":17348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8674,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8663","reported_seq":8793,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:26.981863+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:26.981863+0000","last_peered":"2024-09-05T13:15:26.981863+0000","last_clean":"2024-09-05T13:15:26.981863+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:26.981863+0000","last_undegraded":"2024-09-05T13:15:26.981863+0000","last_fullsized":"2024-09-05T13:15:26.981863+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35454976,"num_objects":8656,"num_object_clones":0,"num_object_copies":17312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5203,"num_write_kb":20812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8793","reported_seq":8927,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.105336+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:27.105336+0000","last_peered":"2024-09-05T13:15:27.105336+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:27.105336+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8793,"log_dups_size":0,"ondisk_log_size":8793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4337},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8569","reported_seq":8704,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.119151+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:27.119151+0000","last_peered":"2024-09-05T13:15:27.119151+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:27.119151+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8569,"log_dups_size":0,"ondisk_log_size":8569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074048,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4127},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8768","reported_seq":8895,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:27.052093+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:27.052093+0000","last_peered":"2024-09-05T13:15:27.052093+0000","last_clean":"2024-09-05T13:15:27.052093+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:27.052093+0000","last_undegraded":"2024-09-05T13:15:27.052093+0000","last_fullsized":"2024-09-05T13:15:27.052093+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35905536,"num_objects":8766,"num_object_clones":0,"num_object_copies":17532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5378,"num_write_kb":21512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8663","reported_seq":8793,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.589582+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:27.589582+0000","last_peered":"2024-09-05T13:15:27.589582+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:27.589582+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8663,"log_dups_size":0,"ondisk_log_size":8663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35454976,"num_objects":8656,"num_object_clones":0,"num_object_copies":17312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6317,"num_write_kb":25268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8669","reported_seq":8802,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.142773+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:27.142773+0000","last_peered":"2024-09-05T13:15:27.142773+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:27.142773+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8669,"log_dups_size":0,"ondisk_log_size":8669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35475456,"num_objects":8661,"num_object_clones":0,"num_object_copies":17322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5036,"num_write_kb":20144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4207},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8676","reported_seq":15097,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.664382+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:27.664382+0000","last_peered":"2024-09-05T13:15:27.664382+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:27.664382+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8676,"log_dups_size":0,"ondisk_log_size":8676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35516416,"num_objects":8671,"num_object_clones":0,"num_object_copies":17342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1232,"num_objects_unfound":0,"num_objects_dirty":8671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6271,"num_write_kb":25084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3155,"num_bytes_recovered":12914688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1232},{"shards":"2","objects":7437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8500","reported_seq":8618,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.564903+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:27.564903+0000","last_peered":"2024-09-05T13:15:27.564903+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:27.564903+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8500,"log_dups_size":0,"ondisk_log_size":8500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34791424,"num_objects":8494,"num_object_clones":0,"num_object_copies":16988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":24036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4107}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8657","reported_seq":8792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.103869+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:27.103869+0000","last_peered":"2024-09-05T13:15:27.103869+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:27.103869+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8657,"log_dups_size":0,"ondisk_log_size":8657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4236},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8722","reported_seq":8905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:26.967027+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:26.967027+0000","last_peered":"2024-09-05T13:15:26.967027+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:26.967027+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8722,"log_dups_size":0,"ondisk_log_size":8722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35692544,"num_objects":8714,"num_object_clones":0,"num_object_copies":17428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4621,"num_write_kb":18484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4270},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8757","reported_seq":8891,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.099615+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:27.099615+0000","last_peered":"2024-09-05T13:15:27.099615+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:27.099615+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8757,"log_dups_size":0,"ondisk_log_size":8757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35852288,"num_objects":8753,"num_object_clones":0,"num_object_copies":17506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4887,"num_write_kb":19548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4187},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8843","reported_seq":8976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:27.057652+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:27.057652+0000","last_peered":"2024-09-05T13:15:27.057652+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:27.057652+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8843,"log_dups_size":0,"ondisk_log_size":8843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36172259,"num_objects":8832,"num_object_clones":0,"num_object_copies":17664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5511,"num_write_kb":22044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4267},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568517091,"num_objects":138799,"num_object_clones":0,"num_object_copies":277598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1232,"num_objects_unfound":0,"num_objects_dirty":138799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94598,"num_write_kb":378392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3158,"num_bytes_recovered":12926976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1154166784,"data_stored":1154083685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138880,"ondisk_log_size":138880,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676752,"kb_used_data":420160,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93695088,"statfs":{"total":96636764160,"available":95943770112,"internally_reserved":0,"allocated":430243840,"data_stored":429998750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789952,"kb_used_data":487600,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93581888,"statfs":{"total":96636764160,"available":95827853312,"internally_reserved":0,"allocated":499302400,"data_stored":499053233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416404,"kb_used_data":218116,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93955436,"statfs":{"total":96636764160,"available":96210366464,"internally_reserved":0,"allocated":223350784,"data_stored":223029357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499359744,"data_stored":499356131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224555008,"data_stored":224479135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430252032,"data_stored":430248419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:32.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:32.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8573", 'reported_seq': 8643, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.100903+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:27.100903+0000', 'last_peered': '2024-09-05T13:15:27.100903+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:27.100903+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 8573, 'log_dups_size': 0, 'ondisk_log_size': 8573, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35115008, 'num_objects': 8573, 'num_object_clones': 0, 'num_object_copies': 17146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8573, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8573, 'num_write_kb': 34292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4152}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8768", 'reported_seq': 8891, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.627269+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:27.627269+0000', 'last_peered': '2024-09-05T13:15:27.627269+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:27.627269+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 8768, 'log_dups_size': 0, 'ondisk_log_size': 8768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35897344, 'num_objects': 8764, 'num_object_clones': 0, 'num_object_copies': 17528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6579, 'num_write_kb': 26316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 4272}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8793", 'reported_seq': 8927, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.105336+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:27.105336+0000', 'last_peered': '2024-09-05T13:15:27.105336+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:27.105336+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 8793, 'log_dups_size': 0, 'ondisk_log_size': 8793, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35995648, 'num_objects': 8788, 'num_object_clones': 0, 'num_object_copies': 17576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8788, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5248, 'num_write_kb': 20992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4337}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8569", 'reported_seq': 8704, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.119151+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:27.119151+0000', 'last_peered': '2024-09-05T13:15:27.119151+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:27.119151+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 8569, 'log_dups_size': 0, 'ondisk_log_size': 8569, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35074048, 'num_objects': 8563, 'num_object_clones': 0, 'num_object_copies': 17126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4888, 'num_write_kb': 19552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4127}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8663", 'reported_seq': 8793, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.589582+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:27.589582+0000', 'last_peered': '2024-09-05T13:15:27.589582+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:27.589582+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 8663, 'log_dups_size': 0, 'ondisk_log_size': 8663, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35454976, 'num_objects': 8656, 'num_object_clones': 0, 'num_object_copies': 17312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8656, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6317, 'num_write_kb': 25268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 4205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:32.460 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8669", 'reported_seq': 8802, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.142773+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:27.142773+0000', 'last_peered': '2024-09-05T13:15:27.142773+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:27.142773+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 8669, 'log_dups_size': 0, 'ondisk_log_size': 8669, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35475456, 'num_objects': 8661, 'num_object_clones': 0, 'num_object_copies': 17322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5036, 'num_write_kb': 20144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4207}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8676", 'reported_seq': 15097, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.664382+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:27.664382+0000', 'last_peered': '2024-09-05T13:15:27.664382+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:27.664382+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 8676, 'log_dups_size': 0, 'ondisk_log_size': 8676, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35516416, 'num_objects': 8671, 'num_object_clones': 0, 'num_object_copies': 17342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1232, 'num_objects_unfound': 0, 'num_objects_dirty': 8671, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6271, 'num_write_kb': 25084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3155, 'num_bytes_recovered': 12914688, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1232}, {'shards': '2', 'objects': 7437}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8500", 'reported_seq': 8618, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.564903+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:27.564903+0000', 'last_peered': '2024-09-05T13:15:27.564903+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:27.564903+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 8500, 'log_dups_size': 0, 'ondisk_log_size': 8500, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34791424, 'num_objects': 8494, 'num_object_clones': 0, 'num_object_copies': 16988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8494, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6009, 'num_write_kb': 24036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 4107}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8657", 'reported_seq': 8792, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.103869+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:27.103869+0000', 'last_peered': '2024-09-05T13:15:27.103869+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:27.103869+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 8657, 'log_dups_size': 0, 'ondisk_log_size': 8657, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35450880, 'num_objects': 8655, 'num_object_clones': 0, 'num_object_copies': 17310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8655, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4808, 'num_write_kb': 19232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4236}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8722", 'reported_seq': 8905, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:26.967027+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:26.967027+0000', 'last_peered': '2024-09-05T13:15:26.967027+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:26.967027+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 8722, 'log_dups_size': 0, 'ondisk_log_size': 8722, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35692544, 'num_objects': 8714, 'num_object_clones': 0, 'num_object_copies': 17428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8714, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4621, 'num_write_kb': 18484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4270}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:32.461 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:32.462 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8757", 'reported_seq': 8891, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.099615+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:27.099615+0000', 'last_peered': '2024-09-05T13:15:27.099615+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:27.099615+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 8757, 'log_dups_size': 0, 'ondisk_log_size': 8757, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35852288, 'num_objects': 8753, 'num_object_clones': 0, 'num_object_copies': 17506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4887, 'num_write_kb': 19548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4187}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:32.462 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:32.462 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8843", 'reported_seq': 8976, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:27.057652+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:27.057652+0000', 'last_peered': '2024-09-05T13:15:27.057652+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:27.057652+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 8843, 'log_dups_size': 0, 'ondisk_log_size': 8843, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36172259, 'num_objects': 8832, 'num_object_clones': 0, 'num_object_copies': 17664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5511, 'num_write_kb': 22044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4267}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:32.462 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:33.155 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:33.155 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/277602 objects degraded (17.690%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":260,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138801,"data_bytes":568517110,"bytes_used":1928302592,"bytes_avail":287981989888,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":277602,"degraded_ratio":0.17690074279003754,"misplaced_objects":1232,"misplaced_total":277602,"misplaced_ratio":0.0044380083716976104,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":97926,"recovering_keys_per_sec":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"write_bytes_sec":1599916,"read_op_per_sec":0,"write_op_per_sec":390},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:33.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:33.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:33.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:15:33.234087+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0254446 2024-09-05T13:15:33.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:15:33.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 180 20 141496 141476 3.06939 1.66016 0.0613445 0.0254446 2024-09-05T13:15:33.733 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:33.735 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:33.759 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-05T13:15:33.571655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":578019830,"num_objects":141121,"num_object_clones":0,"num_object_copies":282242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":141121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96920,"num_write_kb":387674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3309,"num_bytes_recovered":13541395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141203,"ondisk_log_size":141203,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1886468,"kb_used_data":1137556,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281229052,"statfs":{"total":289910292480,"available":287978549248,"internally_reserved":0,"allocated":1164857344,"data_stored":1164041660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":766823430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,6,4,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-437,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":28036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":1789952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004594"},"pg_stats":[{"pgid":"2.f","version":"43'8737","reported_seq":8805,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.129627+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:32.129627+0000","last_peered":"2024-09-05T13:15:32.129627+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:32.129627+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8737,"log_dups_size":0,"ondisk_log_size":8737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8737,"num_write_kb":34948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4316},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8736","reported_seq":8836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.584980+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:32.584980+0000","last_peered":"2024-09-05T13:15:32.584980+0000","last_clean":"2024-09-05T13:15:32.584980+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:32.584980+0000","last_undegraded":"2024-09-05T13:15:32.584980+0000","last_fullsized":"2024-09-05T13:15:32.584980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8736,"log_dups_size":0,"ondisk_log_size":8736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758080,"num_objects":8730,"num_object_clones":0,"num_object_copies":17460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6746,"num_write_kb":26984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8905","reported_seq":9028,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.653749+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:32.653749+0000","last_peered":"2024-09-05T13:15:32.653749+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:32.653749+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8905,"log_dups_size":0,"ondisk_log_size":8905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36458496,"num_objects":8901,"num_object_clones":0,"num_object_copies":17802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6716,"num_write_kb":26864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8820","reported_seq":8885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.553150+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:32.553150+0000","last_peered":"2024-09-05T13:15:32.553150+0000","last_clean":"2024-09-05T13:15:32.553150+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:32.553150+0000","last_undegraded":"2024-09-05T13:15:32.553150+0000","last_fullsized":"2024-09-05T13:15:32.553150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8820,"log_dups_size":0,"ondisk_log_size":8820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36126720,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8820,"num_write_kb":35280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8807","reported_seq":8937,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.121398+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:32.121398+0000","last_peered":"2024-09-05T13:15:32.121398+0000","last_clean":"2024-09-05T13:15:32.121398+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:32.121398+0000","last_undegraded":"2024-09-05T13:15:32.121398+0000","last_fullsized":"2024-09-05T13:15:32.121398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8807,"log_dups_size":0,"ondisk_log_size":8807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36044800,"num_objects":8800,"num_object_clones":0,"num_object_copies":17600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5347,"num_write_kb":21388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8938","reported_seq":9072,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.130640+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:32.130640+0000","last_peered":"2024-09-05T13:15:32.130640+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:32.130640+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8938,"log_dups_size":0,"ondisk_log_size":8938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36589568,"num_objects":8933,"num_object_clones":0,"num_object_copies":17866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5393,"num_write_kb":21572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4482},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8689","reported_seq":8824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.096974+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:32.096974+0000","last_peered":"2024-09-05T13:15:32.096974+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:32.096974+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8689,"log_dups_size":0,"ondisk_log_size":8689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4247},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8907","reported_seq":9034,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.074892+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:32.074892+0000","last_peered":"2024-09-05T13:15:32.074892+0000","last_clean":"2024-09-05T13:15:32.074892+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:32.074892+0000","last_undegraded":"2024-09-05T13:15:32.074892+0000","last_fullsized":"2024-09-05T13:15:32.074892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8907,"log_dups_size":0,"ondisk_log_size":8907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36474880,"num_objects":8905,"num_object_clones":0,"num_object_copies":17810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5517,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8802","reported_seq":8932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.643753+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:32.643753+0000","last_peered":"2024-09-05T13:15:32.643753+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:32.643753+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024320,"num_objects":8795,"num_object_clones":0,"num_object_copies":17590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8823","reported_seq":8956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.132167+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:32.132167+0000","last_peered":"2024-09-05T13:15:32.132167+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:32.132167+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8823,"log_dups_size":0,"ondisk_log_size":8823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106240,"num_objects":8815,"num_object_clones":0,"num_object_copies":17630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5190,"num_write_kb":20760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4361},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8827","reported_seq":15548,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.674189+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:32.674189+0000","last_peered":"2024-09-05T13:15:32.674189+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:32.674189+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6422,"num_write_kb":25688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3305,"num_bytes_recovered":13529088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1082},{"shards":"2","objects":7738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8648","reported_seq":8766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.615395+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:32.615395+0000","last_peered":"2024-09-05T13:15:32.615395+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:32.615395+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8648,"log_dups_size":0,"ondisk_log_size":8648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35397632,"num_objects":8642,"num_object_clones":0,"num_object_copies":17284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6157,"num_write_kb":24628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8797","reported_seq":8933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.067460+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:32.067460+0000","last_peered":"2024-09-05T13:15:32.067460+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:32.067460+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8797,"log_dups_size":0,"ondisk_log_size":8797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024320,"num_objects":8795,"num_object_clones":0,"num_object_copies":17590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4376},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8869","reported_seq":9052,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.137006+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:32.137006+0000","last_peered":"2024-09-05T13:15:32.137006+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:32.137006+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36294656,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4768,"num_write_kb":19072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4417},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8910","reported_seq":9044,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.164376+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:32.164376+0000","last_peered":"2024-09-05T13:15:32.164376+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:32.164376+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8910,"log_dups_size":0,"ondisk_log_size":8910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36478976,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5040,"num_write_kb":20160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4340},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8985","reported_seq":9120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:31.972594+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:31.972594+0000","last_peered":"2024-09-05T13:15:31.972594+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:31.972594+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8985,"log_dups_size":0,"ondisk_log_size":8985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753891,"num_objects":8974,"num_object_clones":0,"num_object_copies":17948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5653,"num_write_kb":22612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4409},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":578019811,"num_objects":141119,"num_object_clones":0,"num_object_copies":282238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":141119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96918,"num_write_kb":387672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3308,"num_bytes_recovered":13541376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1166024704,"data_stored":1165941605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141200,"ondisk_log_size":141200,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668888,"kb_used_data":424776,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93702952,"statfs":{"total":96636764160,"available":95951822848,"internally_reserved":0,"allocated":434970624,"data_stored":434725534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":801176,"kb_used_data":494664,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93570664,"statfs":{"total":96636764160,"available":95816359936,"internally_reserved":0,"allocated":506535936,"data_stored":506286769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416404,"kb_used_data":218116,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93955436,"statfs":{"total":96636764160,"available":96210366464,"internally_reserved":0,"allocated":223350784,"data_stored":223029357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506482688,"data_stored":506479075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224555008,"data_stored":224479135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434987008,"data_stored":434983395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:34.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 181 20 141953 141933 3.06229 1.78516 0.0328156 0.0255045 2024-09-05T13:15:35.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 182 20 142434 142414 3.05579 1.87891 0.0604007 0.0255587 2024-09-05T13:15:36.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 183 20 142924 142904 3.04955 1.91406 0.0509263 0.0256108 2024-09-05T13:15:36.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:37.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 184 20 143383 143363 3.04272 1.79297 0.0276741 0.0256684 2024-09-05T13:15:37.334 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:37.334 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:37.363 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-05T13:15:35.572508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":578019830,"num_objects":141121,"num_object_clones":0,"num_object_copies":282242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":141121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96920,"num_write_kb":387674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3309,"num_bytes_recovered":13541395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141203,"ondisk_log_size":141203,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898056,"kb_used_data":1144984,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281217464,"statfs":{"total":289910292480,"available":287966683136,"internally_reserved":0,"allocated":1172463616,"data_stored":1171647932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":771083270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,1,6,10,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19079168,"num_objects":4658,"num_object_clones":0,"num_object_copies":9316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":4658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":18632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004615"},"pg_stats":[{"pgid":"2.f","version":"43'8737","reported_seq":8805,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.129627+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:32.129627+0000","last_peered":"2024-09-05T13:15:32.129627+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:32.129627+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8737,"log_dups_size":0,"ondisk_log_size":8737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35786752,"num_objects":8737,"num_object_clones":0,"num_object_copies":17474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8737,"num_write_kb":34948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4316},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8736","reported_seq":8836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.584980+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:32.584980+0000","last_peered":"2024-09-05T13:15:32.584980+0000","last_clean":"2024-09-05T13:15:32.584980+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:32.584980+0000","last_undegraded":"2024-09-05T13:15:32.584980+0000","last_fullsized":"2024-09-05T13:15:32.584980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8736,"log_dups_size":0,"ondisk_log_size":8736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758080,"num_objects":8730,"num_object_clones":0,"num_object_copies":17460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6746,"num_write_kb":26984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8905","reported_seq":9028,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.653749+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:32.653749+0000","last_peered":"2024-09-05T13:15:32.653749+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:32.653749+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8905,"log_dups_size":0,"ondisk_log_size":8905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36458496,"num_objects":8901,"num_object_clones":0,"num_object_copies":17802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6716,"num_write_kb":26864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8820","reported_seq":8885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.553150+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:32.553150+0000","last_peered":"2024-09-05T13:15:32.553150+0000","last_clean":"2024-09-05T13:15:32.553150+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:32.553150+0000","last_undegraded":"2024-09-05T13:15:32.553150+0000","last_fullsized":"2024-09-05T13:15:32.553150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8820,"log_dups_size":0,"ondisk_log_size":8820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36126720,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8820,"num_write_kb":35280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8807","reported_seq":8937,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.121398+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:32.121398+0000","last_peered":"2024-09-05T13:15:32.121398+0000","last_clean":"2024-09-05T13:15:32.121398+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:32.121398+0000","last_undegraded":"2024-09-05T13:15:32.121398+0000","last_fullsized":"2024-09-05T13:15:32.121398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8807,"log_dups_size":0,"ondisk_log_size":8807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36044800,"num_objects":8800,"num_object_clones":0,"num_object_copies":17600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5347,"num_write_kb":21388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8938","reported_seq":9072,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.130640+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:32.130640+0000","last_peered":"2024-09-05T13:15:32.130640+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:32.130640+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":8938,"log_dups_size":0,"ondisk_log_size":8938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36589568,"num_objects":8933,"num_object_clones":0,"num_object_copies":17866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5393,"num_write_kb":21572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4482},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8689","reported_seq":8824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.096974+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:32.096974+0000","last_peered":"2024-09-05T13:15:32.096974+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:32.096974+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8689,"log_dups_size":0,"ondisk_log_size":8689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35565568,"num_objects":8683,"num_object_clones":0,"num_object_copies":17366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4247},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8907","reported_seq":9034,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.074892+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:32.074892+0000","last_peered":"2024-09-05T13:15:32.074892+0000","last_clean":"2024-09-05T13:15:32.074892+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:32.074892+0000","last_undegraded":"2024-09-05T13:15:32.074892+0000","last_fullsized":"2024-09-05T13:15:32.074892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":8907,"log_dups_size":0,"ondisk_log_size":8907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36474880,"num_objects":8905,"num_object_clones":0,"num_object_copies":17810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5517,"num_write_kb":22068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8802","reported_seq":8932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.643753+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:32.643753+0000","last_peered":"2024-09-05T13:15:32.643753+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:32.643753+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024320,"num_objects":8795,"num_object_clones":0,"num_object_copies":17590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8823","reported_seq":8956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.132167+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:32.132167+0000","last_peered":"2024-09-05T13:15:32.132167+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:32.132167+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8823,"log_dups_size":0,"ondisk_log_size":8823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106240,"num_objects":8815,"num_object_clones":0,"num_object_copies":17630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5190,"num_write_kb":20760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4361},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8827","reported_seq":15548,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.674189+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:32.674189+0000","last_peered":"2024-09-05T13:15:32.674189+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:32.674189+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6422,"num_write_kb":25688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3305,"num_bytes_recovered":13529088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1082},{"shards":"2","objects":7738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8648","reported_seq":8766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.615395+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:32.615395+0000","last_peered":"2024-09-05T13:15:32.615395+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:32.615395+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8648,"log_dups_size":0,"ondisk_log_size":8648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35397632,"num_objects":8642,"num_object_clones":0,"num_object_copies":17284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6157,"num_write_kb":24628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8797","reported_seq":8933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.067460+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:32.067460+0000","last_peered":"2024-09-05T13:15:32.067460+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:32.067460+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8797,"log_dups_size":0,"ondisk_log_size":8797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024320,"num_objects":8795,"num_object_clones":0,"num_object_copies":17590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4948,"num_write_kb":19792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4376},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8869","reported_seq":9052,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.137006+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:32.137006+0000","last_peered":"2024-09-05T13:15:32.137006+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:32.137006+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36294656,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4768,"num_write_kb":19072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4417},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8910","reported_seq":9044,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.164376+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:32.164376+0000","last_peered":"2024-09-05T13:15:32.164376+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:32.164376+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":8910,"log_dups_size":0,"ondisk_log_size":8910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36478976,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5040,"num_write_kb":20160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4340},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8985","reported_seq":9120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:31.972594+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:31.972594+0000","last_peered":"2024-09-05T13:15:31.972594+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:31.972594+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":8985,"log_dups_size":0,"ondisk_log_size":8985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753891,"num_objects":8974,"num_object_clones":0,"num_object_copies":17948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5653,"num_write_kb":22612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4409},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":578019811,"num_objects":141119,"num_object_clones":0,"num_object_copies":282238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":141119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96918,"num_write_kb":387672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3308,"num_bytes_recovered":13541376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1173676032,"data_stored":1173592933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141200,"ondisk_log_size":141200,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668888,"kb_used_data":424776,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93702952,"statfs":{"total":96636764160,"available":95951822848,"internally_reserved":0,"allocated":434970624,"data_stored":434725534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":801176,"kb_used_data":494664,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93570664,"statfs":{"total":96636764160,"available":95816359936,"internally_reserved":0,"allocated":506535936,"data_stored":506286769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427992,"kb_used_data":225544,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93943848,"statfs":{"total":96636764160,"available":96198500352,"internally_reserved":0,"allocated":230957056,"data_stored":230635629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207285935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506482688,"data_stored":506479075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232206336,"data_stored":232130463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434987008,"data_stored":434983395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:37.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:37.934 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:37.965 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-05T13:15:37.573153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583889398,"num_objects":142554,"num_object_clones":0,"num_object_copies":285108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":142554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98353,"num_write_kb":393406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3309,"num_bytes_recovered":13541395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142636,"ondisk_log_size":142636,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1909116,"kb_used_data":1151884,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281206404,"statfs":{"total":289910292480,"available":287955357696,"internally_reserved":0,"allocated":1179529216,"data_stored":1178713532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":775343110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,3,4,4,12,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24948736,"num_objects":6091,"num_object_clones":0,"num_object_copies":12182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":6091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6091,"num_write_kb":24364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004442"},"pg_stats":[{"pgid":"2.f","version":"43'8876","reported_seq":8945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.158637+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:37.158637+0000","last_peered":"2024-09-05T13:15:37.158637+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:37.158637+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8876,"log_dups_size":0,"ondisk_log_size":8876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36356096,"num_objects":8876,"num_object_clones":0,"num_object_copies":17752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8876,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8736","reported_seq":8836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.584980+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:32.584980+0000","last_peered":"2024-09-05T13:15:32.584980+0000","last_clean":"2024-09-05T13:15:32.584980+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:32.584980+0000","last_undegraded":"2024-09-05T13:15:32.584980+0000","last_fullsized":"2024-09-05T13:15:32.584980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8736,"log_dups_size":0,"ondisk_log_size":8736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758080,"num_objects":8730,"num_object_clones":0,"num_object_copies":17460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6746,"num_write_kb":26984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8905","reported_seq":9028,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.653749+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:32.653749+0000","last_peered":"2024-09-05T13:15:32.653749+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:32.653749+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8905,"log_dups_size":0,"ondisk_log_size":8905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36458496,"num_objects":8901,"num_object_clones":0,"num_object_copies":17802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6716,"num_write_kb":26864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8820","reported_seq":8885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.553150+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:32.553150+0000","last_peered":"2024-09-05T13:15:32.553150+0000","last_clean":"2024-09-05T13:15:32.553150+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:32.553150+0000","last_undegraded":"2024-09-05T13:15:32.553150+0000","last_fullsized":"2024-09-05T13:15:32.553150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8820,"log_dups_size":0,"ondisk_log_size":8820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36126720,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8820,"num_write_kb":35280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8974","reported_seq":9104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.114127+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:37.114127+0000","last_peered":"2024-09-05T13:15:37.114127+0000","last_clean":"2024-09-05T13:15:37.114127+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:37.114127+0000","last_undegraded":"2024-09-05T13:15:37.114127+0000","last_fullsized":"2024-09-05T13:15:37.114127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8974,"log_dups_size":0,"ondisk_log_size":8974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5514,"num_write_kb":22056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9068","reported_seq":9202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.042960+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:37.042960+0000","last_peered":"2024-09-05T13:15:37.042960+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:37.042960+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9068,"log_dups_size":0,"ondisk_log_size":9068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122048,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5523,"num_write_kb":22092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8832","reported_seq":8965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.206426+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:37.206426+0000","last_peered":"2024-09-05T13:15:37.206426+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:37.206426+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8832,"log_dups_size":0,"ondisk_log_size":8832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4390},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9064","reported_seq":9191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.143176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:37.143176+0000","last_peered":"2024-09-05T13:15:37.143176+0000","last_clean":"2024-09-05T13:15:37.143176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:37.143176+0000","last_undegraded":"2024-09-05T13:15:37.143176+0000","last_fullsized":"2024-09-05T13:15:37.143176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9064,"log_dups_size":0,"ondisk_log_size":9064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37117952,"num_objects":9062,"num_object_clones":0,"num_object_copies":18124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5674,"num_write_kb":22696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8802","reported_seq":8932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.643753+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:32.643753+0000","last_peered":"2024-09-05T13:15:32.643753+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:32.643753+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024320,"num_objects":8795,"num_object_clones":0,"num_object_copies":17590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8957","reported_seq":9090,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.171734+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:37.171734+0000","last_peered":"2024-09-05T13:15:37.171734+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:37.171734+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8957,"log_dups_size":0,"ondisk_log_size":8957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5324,"num_write_kb":21296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4495},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8827","reported_seq":15548,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.674189+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:32.674189+0000","last_peered":"2024-09-05T13:15:32.674189+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:32.674189+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6422,"num_write_kb":25688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3305,"num_bytes_recovered":13529088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1082},{"shards":"2","objects":7738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8648","reported_seq":8766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.615395+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:32.615395+0000","last_peered":"2024-09-05T13:15:32.615395+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:32.615395+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8648,"log_dups_size":0,"ondisk_log_size":8648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35397632,"num_objects":8642,"num_object_clones":0,"num_object_copies":17284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6157,"num_write_kb":24628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8937","reported_seq":9073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.183576+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:37.183576+0000","last_peered":"2024-09-05T13:15:37.183576+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:37.183576+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5088,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9026","reported_seq":9209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.138853+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:37.138853+0000","last_peered":"2024-09-05T13:15:37.138853+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:37.138853+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9026,"log_dups_size":0,"ondisk_log_size":9026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4925,"num_write_kb":19700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4574},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9041","reported_seq":9174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.185484+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:37.185484+0000","last_peered":"2024-09-05T13:15:37.185484+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:37.185484+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9041,"log_dups_size":0,"ondisk_log_size":9041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37015552,"num_objects":9037,"num_object_clones":0,"num_object_copies":18074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5171,"num_write_kb":20684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4471},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9120","reported_seq":9255,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.196864+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:37.196864+0000","last_peered":"2024-09-05T13:15:37.196864+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:37.196864+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9120,"log_dups_size":0,"ondisk_log_size":9120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37306851,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5788,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4544},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583889379,"num_objects":142552,"num_object_clones":0,"num_object_copies":285104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":142552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98351,"num_write_kb":393404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3308,"num_bytes_recovered":13541376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180737536,"data_stored":1180654437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142633,"ondisk_log_size":142633,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668888,"kb_used_data":424776,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93702952,"statfs":{"total":96636764160,"available":95951822848,"internally_reserved":0,"allocated":434970624,"data_stored":434725534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":812236,"kb_used_data":501564,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93559604,"statfs":{"total":96636764160,"available":95805034496,"internally_reserved":0,"allocated":513601536,"data_stored":513352369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427992,"kb_used_data":225544,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93943848,"statfs":{"total":96636764160,"available":96198500352,"internally_reserved":0,"allocated":230957056,"data_stored":230635629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207285935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513544192,"data_stored":513540579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232206336,"data_stored":232130463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434987008,"data_stored":434983395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:37.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:37.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8876", 'reported_seq': 8945, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.158637+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:37.158637+0000', 'last_peered': '2024-09-05T13:15:37.158637+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:37.158637+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 8876, 'log_dups_size': 0, 'ondisk_log_size': 8876, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36356096, 'num_objects': 8876, 'num_object_clones': 0, 'num_object_copies': 17752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8876, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8876, 'num_write_kb': 35504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4455}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:37.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:37.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8905", 'reported_seq': 9028, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:32.653749+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:32.653749+0000', 'last_peered': '2024-09-05T13:15:32.653749+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:32.653749+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 8905, 'log_dups_size': 0, 'ondisk_log_size': 8905, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36458496, 'num_objects': 8901, 'num_object_clones': 0, 'num_object_copies': 17802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6716, 'num_write_kb': 26864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 4409}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:37.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'9068", 'reported_seq': 9202, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.042960+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:37.042960+0000', 'last_peered': '2024-09-05T13:15:37.042960+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:37.042960+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 9068, 'log_dups_size': 0, 'ondisk_log_size': 9068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37122048, 'num_objects': 9063, 'num_object_clones': 0, 'num_object_copies': 18126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9063, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5523, 'num_write_kb': 22092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4612}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8832", 'reported_seq': 8965, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.206426+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:37.206426+0000', 'last_peered': '2024-09-05T13:15:37.206426+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:37.206426+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 8832, 'log_dups_size': 0, 'ondisk_log_size': 8832, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36151296, 'num_objects': 8826, 'num_object_clones': 0, 'num_object_copies': 17652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5151, 'num_write_kb': 20604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4390}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8802", 'reported_seq': 8932, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:32.643753+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:32.643753+0000', 'last_peered': '2024-09-05T13:15:32.643753+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:32.643753+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 8802, 'log_dups_size': 0, 'ondisk_log_size': 8802, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36024320, 'num_objects': 8795, 'num_object_clones': 0, 'num_object_copies': 17590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8795, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6456, 'num_write_kb': 25824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 4344}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8957", 'reported_seq': 9090, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.171734+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:37.171734+0000', 'last_peered': '2024-09-05T13:15:37.171734+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:37.171734+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 8957, 'log_dups_size': 0, 'ondisk_log_size': 8957, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36655104, 'num_objects': 8949, 'num_object_clones': 0, 'num_object_copies': 17898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5324, 'num_write_kb': 21296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4495}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8827", 'reported_seq': 15548, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:32.674189+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:32.674189+0000', 'last_peered': '2024-09-05T13:15:32.674189+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:32.674189+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 8827, 'log_dups_size': 0, 'ondisk_log_size': 8827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36134912, 'num_objects': 8822, 'num_object_clones': 0, 'num_object_copies': 17644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1082, 'num_objects_unfound': 0, 'num_objects_dirty': 8822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6422, 'num_write_kb': 25688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3305, 'num_bytes_recovered': 13529088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1082}, {'shards': '2', 'objects': 7738}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:37.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8648", 'reported_seq': 8766, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:32.615395+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:32.615395+0000', 'last_peered': '2024-09-05T13:15:32.615395+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:32.615395+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 8648, 'log_dups_size': 0, 'ondisk_log_size': 8648, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35397632, 'num_objects': 8642, 'num_object_clones': 0, 'num_object_copies': 17284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8642, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6157, 'num_write_kb': 24628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 4255}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8937", 'reported_seq': 9073, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.183576+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:37.183576+0000', 'last_peered': '2024-09-05T13:15:37.183576+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:37.183576+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 8937, 'log_dups_size': 0, 'ondisk_log_size': 8937, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36597760, 'num_objects': 8935, 'num_object_clones': 0, 'num_object_copies': 17870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5088, 'num_write_kb': 20352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4516}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'9026", 'reported_seq': 9209, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.138853+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:37.138853+0000', 'last_peered': '2024-09-05T13:15:37.138853+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:37.138853+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 9026, 'log_dups_size': 0, 'ondisk_log_size': 9026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36937728, 'num_objects': 9018, 'num_object_clones': 0, 'num_object_copies': 18036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4925, 'num_write_kb': 19700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4574}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'9041", 'reported_seq': 9174, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.185484+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:37.185484+0000', 'last_peered': '2024-09-05T13:15:37.185484+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:37.185484+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 9041, 'log_dups_size': 0, 'ondisk_log_size': 9041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37015552, 'num_objects': 9037, 'num_object_clones': 0, 'num_object_copies': 18074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5171, 'num_write_kb': 20684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4471}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:37.968 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'9120", 'reported_seq': 9255, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.196864+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:37.196864+0000', 'last_peered': '2024-09-05T13:15:37.196864+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:37.196864+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 9120, 'log_dups_size': 0, 'ondisk_log_size': 9120, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37306851, 'num_objects': 9109, 'num_object_clones': 0, 'num_object_copies': 18218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5788, 'num_write_kb': 23152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4544}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:37.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:38.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 185 20 143819 143799 3.03548 1.70312 0.0333517 0.0257315 2024-09-05T13:15:38.648 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:38.648 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/285108 objects degraded (17.224%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":265,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142554,"data_bytes":583889398,"bytes_used":1954934784,"bytes_avail":287955357696,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":285108,"degraded_ratio":0.17224350070850344,"misplaced_objects":1082,"misplaced_total":285108,"misplaced_ratio":0.0037950531026838953,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":102020,"recovering_keys_per_sec":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"write_bytes_sec":2078291,"read_op_per_sec":0,"write_op_per_sec":507},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:38.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:38.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:39.216 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:39.217 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:39.234 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 186 20 144236 144216 3.02792 1.62891 0.0471653 0.0257923 2024-09-05T13:15:39.242 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-05T13:15:37.573153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583889398,"num_objects":142554,"num_object_clones":0,"num_object_copies":285108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":142554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98353,"num_write_kb":393406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3309,"num_bytes_recovered":13541395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142636,"ondisk_log_size":142636,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1909116,"kb_used_data":1151884,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281206404,"statfs":{"total":289910292480,"available":287955357696,"internally_reserved":0,"allocated":1179529216,"data_stored":1178713532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":775343110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,3,4,4,12,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24948736,"num_objects":6091,"num_object_clones":0,"num_object_copies":12182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":6091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6091,"num_write_kb":24364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004442"},"pg_stats":[{"pgid":"2.f","version":"43'8876","reported_seq":8945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.158637+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:37.158637+0000","last_peered":"2024-09-05T13:15:37.158637+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:37.158637+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8876,"log_dups_size":0,"ondisk_log_size":8876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36356096,"num_objects":8876,"num_object_clones":0,"num_object_copies":17752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8876,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8736","reported_seq":8836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.584980+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:32.584980+0000","last_peered":"2024-09-05T13:15:32.584980+0000","last_clean":"2024-09-05T13:15:32.584980+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:32.584980+0000","last_undegraded":"2024-09-05T13:15:32.584980+0000","last_fullsized":"2024-09-05T13:15:32.584980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8736,"log_dups_size":0,"ondisk_log_size":8736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758080,"num_objects":8730,"num_object_clones":0,"num_object_copies":17460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6746,"num_write_kb":26984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8905","reported_seq":9028,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.653749+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:32.653749+0000","last_peered":"2024-09-05T13:15:32.653749+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:32.653749+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":8905,"log_dups_size":0,"ondisk_log_size":8905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36458496,"num_objects":8901,"num_object_clones":0,"num_object_copies":17802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6716,"num_write_kb":26864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8820","reported_seq":8885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:32.553150+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:32.553150+0000","last_peered":"2024-09-05T13:15:32.553150+0000","last_clean":"2024-09-05T13:15:32.553150+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:32.553150+0000","last_undegraded":"2024-09-05T13:15:32.553150+0000","last_fullsized":"2024-09-05T13:15:32.553150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8820,"log_dups_size":0,"ondisk_log_size":8820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36126720,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8820,"num_write_kb":35280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8974","reported_seq":9104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.114127+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:37.114127+0000","last_peered":"2024-09-05T13:15:37.114127+0000","last_clean":"2024-09-05T13:15:37.114127+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:37.114127+0000","last_undegraded":"2024-09-05T13:15:37.114127+0000","last_fullsized":"2024-09-05T13:15:37.114127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8974,"log_dups_size":0,"ondisk_log_size":8974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5514,"num_write_kb":22056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9068","reported_seq":9202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.042960+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:37.042960+0000","last_peered":"2024-09-05T13:15:37.042960+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:37.042960+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9068,"log_dups_size":0,"ondisk_log_size":9068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122048,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5523,"num_write_kb":22092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8832","reported_seq":8965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.206426+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:37.206426+0000","last_peered":"2024-09-05T13:15:37.206426+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:37.206426+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8832,"log_dups_size":0,"ondisk_log_size":8832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4390},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9064","reported_seq":9191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.143176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:37.143176+0000","last_peered":"2024-09-05T13:15:37.143176+0000","last_clean":"2024-09-05T13:15:37.143176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:37.143176+0000","last_undegraded":"2024-09-05T13:15:37.143176+0000","last_fullsized":"2024-09-05T13:15:37.143176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9064,"log_dups_size":0,"ondisk_log_size":9064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37117952,"num_objects":9062,"num_object_clones":0,"num_object_copies":18124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5674,"num_write_kb":22696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8802","reported_seq":8932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.643753+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:32.643753+0000","last_peered":"2024-09-05T13:15:32.643753+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:32.643753+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024320,"num_objects":8795,"num_object_clones":0,"num_object_copies":17590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6456,"num_write_kb":25824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8957","reported_seq":9090,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.171734+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:37.171734+0000","last_peered":"2024-09-05T13:15:37.171734+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:37.171734+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8957,"log_dups_size":0,"ondisk_log_size":8957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5324,"num_write_kb":21296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4495},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8827","reported_seq":15548,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.674189+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:32.674189+0000","last_peered":"2024-09-05T13:15:32.674189+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:32.674189+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6422,"num_write_kb":25688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3305,"num_bytes_recovered":13529088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1082},{"shards":"2","objects":7738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8648","reported_seq":8766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:32.615395+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:32.615395+0000","last_peered":"2024-09-05T13:15:32.615395+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:32.615395+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8648,"log_dups_size":0,"ondisk_log_size":8648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35397632,"num_objects":8642,"num_object_clones":0,"num_object_copies":17284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6157,"num_write_kb":24628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8937","reported_seq":9073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.183576+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:37.183576+0000","last_peered":"2024-09-05T13:15:37.183576+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:37.183576+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5088,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9026","reported_seq":9209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.138853+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:37.138853+0000","last_peered":"2024-09-05T13:15:37.138853+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:37.138853+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9026,"log_dups_size":0,"ondisk_log_size":9026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4925,"num_write_kb":19700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4574},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9041","reported_seq":9174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.185484+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:37.185484+0000","last_peered":"2024-09-05T13:15:37.185484+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:37.185484+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9041,"log_dups_size":0,"ondisk_log_size":9041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37015552,"num_objects":9037,"num_object_clones":0,"num_object_copies":18074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5171,"num_write_kb":20684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4471},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9120","reported_seq":9255,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.196864+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:37.196864+0000","last_peered":"2024-09-05T13:15:37.196864+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:37.196864+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9120,"log_dups_size":0,"ondisk_log_size":9120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37306851,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5788,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4544},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583889379,"num_objects":142552,"num_object_clones":0,"num_object_copies":285104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":1082,"num_objects_unfound":0,"num_objects_dirty":142552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98351,"num_write_kb":393404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3308,"num_bytes_recovered":13541376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180737536,"data_stored":1180654437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142633,"ondisk_log_size":142633,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668888,"kb_used_data":424776,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93702952,"statfs":{"total":96636764160,"available":95951822848,"internally_reserved":0,"allocated":434970624,"data_stored":434725534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":812236,"kb_used_data":501564,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93559604,"statfs":{"total":96636764160,"available":95805034496,"internally_reserved":0,"allocated":513601536,"data_stored":513352369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427992,"kb_used_data":225544,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93943848,"statfs":{"total":96636764160,"available":96198500352,"internally_reserved":0,"allocated":230957056,"data_stored":230635629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207285935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513544192,"data_stored":513540579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232206336,"data_stored":232130463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434987008,"data_stored":434983395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:40.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 187 20 144723 144703 3.0219 1.90234 0.0396038 0.0258452 2024-09-05T13:15:41.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 188 20 145199 145179 3.01572 1.85938 0.0846543 0.0258981 2024-09-05T13:15:42.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 189 20 145681 145661 3.00973 1.88281 0.0418775 0.0259493 2024-09-05T13:15:42.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:42.797 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:42.799 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:42.827 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-05T13:15:41.574792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":587559414,"num_objects":143450,"num_object_clones":0,"num_object_copies":286900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":931,"num_objects_unfound":0,"num_objects_dirty":143450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99249,"num_write_kb":396990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3460,"num_bytes_recovered":14159891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143532,"ondisk_log_size":143532,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1918736,"kb_used_data":1165664,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281196784,"statfs":{"total":289910292480,"available":287945506816,"internally_reserved":0,"allocated":1193639936,"data_stored":1192824252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":771083270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,3,6,6,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19042304,"num_objects":4649,"num_object_clones":0,"num_object_copies":9298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":4649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4649,"num_write_kb":18596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":1232896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004759"},"pg_stats":[{"pgid":"2.f","version":"43'8876","reported_seq":8945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.158637+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:37.158637+0000","last_peered":"2024-09-05T13:15:37.158637+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:37.158637+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8876,"log_dups_size":0,"ondisk_log_size":8876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36356096,"num_objects":8876,"num_object_clones":0,"num_object_copies":17752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8876,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8878","reported_seq":8976,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.695228+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:37.695228+0000","last_peered":"2024-09-05T13:15:37.695228+0000","last_clean":"2024-09-05T13:15:37.695228+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:37.695228+0000","last_undegraded":"2024-09-05T13:15:37.695228+0000","last_fullsized":"2024-09-05T13:15:37.695228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8878,"log_dups_size":0,"ondisk_log_size":8878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36339712,"num_objects":8872,"num_object_clones":0,"num_object_copies":17744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6888,"num_write_kb":27552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9061","reported_seq":9184,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.587307+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:37.587307+0000","last_peered":"2024-09-05T13:15:37.587307+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:37.587307+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9061,"log_dups_size":0,"ondisk_log_size":9061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":27488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8963","reported_seq":9028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.639332+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:37.639332+0000","last_peered":"2024-09-05T13:15:37.639332+0000","last_clean":"2024-09-05T13:15:37.639332+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:37.639332+0000","last_undegraded":"2024-09-05T13:15:37.639332+0000","last_fullsized":"2024-09-05T13:15:37.639332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8963,"log_dups_size":0,"ondisk_log_size":8963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8974","reported_seq":9104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.114127+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:37.114127+0000","last_peered":"2024-09-05T13:15:37.114127+0000","last_clean":"2024-09-05T13:15:37.114127+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:37.114127+0000","last_undegraded":"2024-09-05T13:15:37.114127+0000","last_fullsized":"2024-09-05T13:15:37.114127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8974,"log_dups_size":0,"ondisk_log_size":8974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5514,"num_write_kb":22056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9068","reported_seq":9202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.042960+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:37.042960+0000","last_peered":"2024-09-05T13:15:37.042960+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:37.042960+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9068,"log_dups_size":0,"ondisk_log_size":9068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122048,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5523,"num_write_kb":22092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8832","reported_seq":8965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.206426+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:37.206426+0000","last_peered":"2024-09-05T13:15:37.206426+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:37.206426+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8832,"log_dups_size":0,"ondisk_log_size":8832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4390},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9064","reported_seq":9191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.143176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:37.143176+0000","last_peered":"2024-09-05T13:15:37.143176+0000","last_clean":"2024-09-05T13:15:37.143176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:37.143176+0000","last_undegraded":"2024-09-05T13:15:37.143176+0000","last_fullsized":"2024-09-05T13:15:37.143176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9064,"log_dups_size":0,"ondisk_log_size":9064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37117952,"num_objects":9062,"num_object_clones":0,"num_object_copies":18124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5674,"num_write_kb":22696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8966","reported_seq":9096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.638520+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:37.638520+0000","last_peered":"2024-09-05T13:15:37.638520+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:37.638520+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8966,"log_dups_size":0,"ondisk_log_size":8966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36696064,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6620,"num_write_kb":26480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8957","reported_seq":9090,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.171734+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:37.171734+0000","last_peered":"2024-09-05T13:15:37.171734+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:37.171734+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8957,"log_dups_size":0,"ondisk_log_size":8957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5324,"num_write_kb":21296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4495},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8976","reported_seq":15999,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.681947+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:37.681947+0000","last_peered":"2024-09-05T13:15:37.681947+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:37.681947+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8976,"log_dups_size":0,"ondisk_log_size":8976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745216,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":931,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6571,"num_write_kb":26284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3456,"num_bytes_recovered":14147584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":931},{"shards":"2","objects":8038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8790","reported_seq":8908,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.635735+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:37.635735+0000","last_peered":"2024-09-05T13:15:37.635735+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:37.635735+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8790,"log_dups_size":0,"ondisk_log_size":8790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35979264,"num_objects":8784,"num_object_clones":0,"num_object_copies":17568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6299,"num_write_kb":25196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4397}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8937","reported_seq":9073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.183576+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:37.183576+0000","last_peered":"2024-09-05T13:15:37.183576+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:37.183576+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5088,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9026","reported_seq":9209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.138853+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:37.138853+0000","last_peered":"2024-09-05T13:15:37.138853+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:37.138853+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9026,"log_dups_size":0,"ondisk_log_size":9026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4925,"num_write_kb":19700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4574},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9041","reported_seq":9174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.185484+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:37.185484+0000","last_peered":"2024-09-05T13:15:37.185484+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:37.185484+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9041,"log_dups_size":0,"ondisk_log_size":9041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37015552,"num_objects":9037,"num_object_clones":0,"num_object_copies":18074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5171,"num_write_kb":20684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4471},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9120","reported_seq":9255,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.196864+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:37.196864+0000","last_peered":"2024-09-05T13:15:37.196864+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:37.196864+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9120,"log_dups_size":0,"ondisk_log_size":9120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37306851,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5788,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4544},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":587559395,"num_objects":143448,"num_object_clones":0,"num_object_copies":286896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":931,"num_objects_unfound":0,"num_objects_dirty":143448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99247,"num_write_kb":396988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3459,"num_bytes_recovered":14159872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1193361408,"data_stored":1193278309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143529,"ondisk_log_size":143529,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677884,"kb_used_data":429612,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93693956,"statfs":{"total":96636764160,"available":95942610944,"internally_reserved":0,"allocated":439922688,"data_stored":439677598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":812236,"kb_used_data":501564,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93559604,"statfs":{"total":96636764160,"available":95805034496,"internally_reserved":0,"allocated":513601536,"data_stored":513352369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428616,"kb_used_data":234488,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93943224,"statfs":{"total":96636764160,"available":96197861376,"internally_reserved":0,"allocated":240115712,"data_stored":239794285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198766255},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513544192,"data_stored":513540579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239828992,"data_stored":239753119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439988224,"data_stored":439984611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:42.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:43.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 190 20 146100 146080 3.0025 1.63672 0.0177378 0.0260109 2024-09-05T13:15:43.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:43.427 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:43.450 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-05T13:15:41.574792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":587559414,"num_objects":143450,"num_object_clones":0,"num_object_copies":286900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":931,"num_objects_unfound":0,"num_objects_dirty":143450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99249,"num_write_kb":396990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3460,"num_bytes_recovered":14159891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143532,"ondisk_log_size":143532,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1918736,"kb_used_data":1165664,"kb_used_omap":12,"kb_used_meta":753011,"kb_avail":281196784,"statfs":{"total":289910292480,"available":287945506816,"internally_reserved":0,"allocated":1193639936,"data_stored":1192824252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":771083270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,3,6,6,9,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19042304,"num_objects":4649,"num_object_clones":0,"num_object_copies":9298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":4649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4649,"num_write_kb":18596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":1232896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004759"},"pg_stats":[{"pgid":"2.f","version":"43'8876","reported_seq":8945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.158637+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:37.158637+0000","last_peered":"2024-09-05T13:15:37.158637+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:37.158637+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8876,"log_dups_size":0,"ondisk_log_size":8876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36356096,"num_objects":8876,"num_object_clones":0,"num_object_copies":17752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8876,"num_write_kb":35504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8878","reported_seq":8976,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.695228+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:37.695228+0000","last_peered":"2024-09-05T13:15:37.695228+0000","last_clean":"2024-09-05T13:15:37.695228+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:37.695228+0000","last_undegraded":"2024-09-05T13:15:37.695228+0000","last_fullsized":"2024-09-05T13:15:37.695228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":8878,"log_dups_size":0,"ondisk_log_size":8878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36339712,"num_objects":8872,"num_object_clones":0,"num_object_copies":17744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6888,"num_write_kb":27552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9061","reported_seq":9184,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.587307+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:37.587307+0000","last_peered":"2024-09-05T13:15:37.587307+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:37.587307+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9061,"log_dups_size":0,"ondisk_log_size":9061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":27488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8963","reported_seq":9028,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.639332+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:37.639332+0000","last_peered":"2024-09-05T13:15:37.639332+0000","last_clean":"2024-09-05T13:15:37.639332+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:37.639332+0000","last_undegraded":"2024-09-05T13:15:37.639332+0000","last_fullsized":"2024-09-05T13:15:37.639332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":8963,"log_dups_size":0,"ondisk_log_size":8963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8974","reported_seq":9104,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.114127+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:37.114127+0000","last_peered":"2024-09-05T13:15:37.114127+0000","last_clean":"2024-09-05T13:15:37.114127+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:37.114127+0000","last_undegraded":"2024-09-05T13:15:37.114127+0000","last_fullsized":"2024-09-05T13:15:37.114127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":8974,"log_dups_size":0,"ondisk_log_size":8974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5514,"num_write_kb":22056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9068","reported_seq":9202,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.042960+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:37.042960+0000","last_peered":"2024-09-05T13:15:37.042960+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:37.042960+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9068,"log_dups_size":0,"ondisk_log_size":9068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122048,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5523,"num_write_kb":22092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4612},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8832","reported_seq":8965,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.206426+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:37.206426+0000","last_peered":"2024-09-05T13:15:37.206426+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:37.206426+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8832,"log_dups_size":0,"ondisk_log_size":8832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4390},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9064","reported_seq":9191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:37.143176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:37.143176+0000","last_peered":"2024-09-05T13:15:37.143176+0000","last_clean":"2024-09-05T13:15:37.143176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:37.143176+0000","last_undegraded":"2024-09-05T13:15:37.143176+0000","last_fullsized":"2024-09-05T13:15:37.143176+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9064,"log_dups_size":0,"ondisk_log_size":9064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37117952,"num_objects":9062,"num_object_clones":0,"num_object_copies":18124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5674,"num_write_kb":22696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8966","reported_seq":9096,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.638520+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:37.638520+0000","last_peered":"2024-09-05T13:15:37.638520+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:37.638520+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":8966,"log_dups_size":0,"ondisk_log_size":8966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36696064,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6620,"num_write_kb":26480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8957","reported_seq":9090,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.171734+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:37.171734+0000","last_peered":"2024-09-05T13:15:37.171734+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:37.171734+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":8957,"log_dups_size":0,"ondisk_log_size":8957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5324,"num_write_kb":21296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4495},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8976","reported_seq":15999,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.681947+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:37.681947+0000","last_peered":"2024-09-05T13:15:37.681947+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:37.681947+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":8976,"log_dups_size":0,"ondisk_log_size":8976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745216,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":931,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6571,"num_write_kb":26284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3456,"num_bytes_recovered":14147584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":931},{"shards":"2","objects":8038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8790","reported_seq":8908,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.635735+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:37.635735+0000","last_peered":"2024-09-05T13:15:37.635735+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:37.635735+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8790,"log_dups_size":0,"ondisk_log_size":8790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35979264,"num_objects":8784,"num_object_clones":0,"num_object_copies":17568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6299,"num_write_kb":25196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4397}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8937","reported_seq":9073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.183576+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:37.183576+0000","last_peered":"2024-09-05T13:15:37.183576+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:37.183576+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":8937,"log_dups_size":0,"ondisk_log_size":8937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36597760,"num_objects":8935,"num_object_clones":0,"num_object_copies":17870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5088,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9026","reported_seq":9209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.138853+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:37.138853+0000","last_peered":"2024-09-05T13:15:37.138853+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:37.138853+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9026,"log_dups_size":0,"ondisk_log_size":9026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36937728,"num_objects":9018,"num_object_clones":0,"num_object_copies":18036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4925,"num_write_kb":19700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4574},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9041","reported_seq":9174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.185484+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:37.185484+0000","last_peered":"2024-09-05T13:15:37.185484+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:37.185484+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9041,"log_dups_size":0,"ondisk_log_size":9041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37015552,"num_objects":9037,"num_object_clones":0,"num_object_copies":18074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5171,"num_write_kb":20684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4471},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9120","reported_seq":9255,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:37.196864+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:37.196864+0000","last_peered":"2024-09-05T13:15:37.196864+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:37.196864+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9120,"log_dups_size":0,"ondisk_log_size":9120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37306851,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5788,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4544},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":587559395,"num_objects":143448,"num_object_clones":0,"num_object_copies":286896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":931,"num_objects_unfound":0,"num_objects_dirty":143448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99247,"num_write_kb":396988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3459,"num_bytes_recovered":14159872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1193361408,"data_stored":1193278309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143529,"ondisk_log_size":143529,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677884,"kb_used_data":429612,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93693956,"statfs":{"total":96636764160,"available":95942610944,"internally_reserved":0,"allocated":439922688,"data_stored":439677598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":812236,"kb_used_data":501564,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93559604,"statfs":{"total":96636764160,"available":95805034496,"internally_reserved":0,"allocated":513601536,"data_stored":513352369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428616,"kb_used_data":234488,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93943224,"statfs":{"total":96636764160,"available":96197861376,"internally_reserved":0,"allocated":240115712,"data_stored":239794285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198766255},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513544192,"data_stored":513540579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239828992,"data_stored":239753119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439988224,"data_stored":439984611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:43.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:43.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8876", 'reported_seq': 8945, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.158637+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:37.158637+0000', 'last_peered': '2024-09-05T13:15:37.158637+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:37.158637+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 8876, 'log_dups_size': 0, 'ondisk_log_size': 8876, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36356096, 'num_objects': 8876, 'num_object_clones': 0, 'num_object_copies': 17752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8876, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8876, 'num_write_kb': 35504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4455}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:43.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:43.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'9061", 'reported_seq': 9184, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.587307+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:37.587307+0000', 'last_peered': '2024-09-05T13:15:37.587307+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:37.587307+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 9061, 'log_dups_size': 0, 'ondisk_log_size': 9061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37097472, 'num_objects': 9057, 'num_object_clones': 0, 'num_object_copies': 18114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9057, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6872, 'num_write_kb': 27488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:43.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:43.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'9068", 'reported_seq': 9202, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.042960+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:37.042960+0000', 'last_peered': '2024-09-05T13:15:37.042960+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:37.042960+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 9068, 'log_dups_size': 0, 'ondisk_log_size': 9068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37122048, 'num_objects': 9063, 'num_object_clones': 0, 'num_object_copies': 18126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9063, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5523, 'num_write_kb': 22092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4612}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8832", 'reported_seq': 8965, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.206426+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:37.206426+0000', 'last_peered': '2024-09-05T13:15:37.206426+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:37.206426+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 8832, 'log_dups_size': 0, 'ondisk_log_size': 8832, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36151296, 'num_objects': 8826, 'num_object_clones': 0, 'num_object_copies': 17652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5151, 'num_write_kb': 20604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4390}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8966", 'reported_seq': 9096, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.638520+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:37.638520+0000', 'last_peered': '2024-09-05T13:15:37.638520+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:37.638520+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 8966, 'log_dups_size': 0, 'ondisk_log_size': 8966, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36696064, 'num_objects': 8959, 'num_object_clones': 0, 'num_object_copies': 17918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8959, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6620, 'num_write_kb': 26480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 4508}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8957", 'reported_seq': 9090, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.171734+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:37.171734+0000', 'last_peered': '2024-09-05T13:15:37.171734+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:37.171734+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 8957, 'log_dups_size': 0, 'ondisk_log_size': 8957, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36655104, 'num_objects': 8949, 'num_object_clones': 0, 'num_object_copies': 17898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5324, 'num_write_kb': 21296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4495}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8976", 'reported_seq': 15999, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.681947+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:37.681947+0000', 'last_peered': '2024-09-05T13:15:37.681947+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:37.681947+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 8976, 'log_dups_size': 0, 'ondisk_log_size': 8976, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36745216, 'num_objects': 8971, 'num_object_clones': 0, 'num_object_copies': 17942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 931, 'num_objects_unfound': 0, 'num_objects_dirty': 8971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6571, 'num_write_kb': 26284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3456, 'num_bytes_recovered': 14147584, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 931}, {'shards': '2', 'objects': 8038}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:43.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8790", 'reported_seq': 8908, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.635735+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:37.635735+0000', 'last_peered': '2024-09-05T13:15:37.635735+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:37.635735+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 8790, 'log_dups_size': 0, 'ondisk_log_size': 8790, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35979264, 'num_objects': 8784, 'num_object_clones': 0, 'num_object_copies': 17568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6299, 'num_write_kb': 25196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 4397}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:43.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:43.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8937", 'reported_seq': 9073, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.183576+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:37.183576+0000', 'last_peered': '2024-09-05T13:15:37.183576+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:37.183576+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 8937, 'log_dups_size': 0, 'ondisk_log_size': 8937, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36597760, 'num_objects': 8935, 'num_object_clones': 0, 'num_object_copies': 17870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5088, 'num_write_kb': 20352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4516}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:43.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:43.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'9026", 'reported_seq': 9209, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.138853+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:37.138853+0000', 'last_peered': '2024-09-05T13:15:37.138853+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:37.138853+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 9026, 'log_dups_size': 0, 'ondisk_log_size': 9026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36937728, 'num_objects': 9018, 'num_object_clones': 0, 'num_object_copies': 18036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4925, 'num_write_kb': 19700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4574}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:43.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:43.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'9041", 'reported_seq': 9174, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.185484+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:37.185484+0000', 'last_peered': '2024-09-05T13:15:37.185484+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:37.185484+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 9041, 'log_dups_size': 0, 'ondisk_log_size': 9041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37015552, 'num_objects': 9037, 'num_object_clones': 0, 'num_object_copies': 18074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5171, 'num_write_kb': 20684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4471}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:43.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:43.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'9120", 'reported_seq': 9255, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:37.196864+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:37.196864+0000', 'last_peered': '2024-09-05T13:15:37.196864+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:37.196864+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 9120, 'log_dups_size': 0, 'ondisk_log_size': 9120, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37306851, 'num_objects': 9109, 'num_object_clones': 0, 'num_object_copies': 18218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5788, 'num_write_kb': 23152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4544}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:43.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:44.147 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:44.147 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/291498 objects degraded (16.847%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":271,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145749,"data_bytes":596976118,"bytes_used":1972162560,"bytes_avail":287938129920,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":291498,"degraded_ratio":0.16846770818324652,"misplaced_objects":778,"misplaced_total":291498,"misplaced_ratio":0.0026689719998078888,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":154861,"recovering_keys_per_sec":0,"num_objects_recovered":454,"num_bytes_recovered":1859584,"num_keys_recovered":0,"write_bytes_sec":2369999,"read_op_per_sec":0,"write_op_per_sec":578},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:44.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:44.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:44.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 191 20 146509 146489 2.99515 1.59766 0.0565152 0.0260749 2024-09-05T13:15:44.734 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:44.734 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:44.762 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-05T13:15:43.578884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596976118,"num_objects":145749,"num_object_clones":0,"num_object_copies":291498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":145749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101548,"num_write_kb":406186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3613,"num_bytes_recovered":14786579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145831,"ondisk_log_size":145831,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1925940,"kb_used_data":1177028,"kb_used_omap":12,"kb_used_meta":748851,"kb_avail":281189580,"statfs":{"total":289910292480,"available":287938129920,"internally_reserved":0,"allocated":1205276672,"data_stored":1204460988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":766823430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,10,1,0,8,11,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28459008,"num_objects":6948,"num_object_clones":0,"num_object_copies":13896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-454,"num_objects_unfound":0,"num_objects_dirty":6948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6948,"num_write_kb":27792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":454,"num_bytes_recovered":1859584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008021"},"pg_stats":[{"pgid":"2.f","version":"43'9041","reported_seq":9109,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.210010+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:42.210010+0000","last_peered":"2024-09-05T13:15:42.210010+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:42.210010+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9041,"log_dups_size":0,"ondisk_log_size":9041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":36164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4620},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9032","reported_seq":9132,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:42.664436+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:42.664436+0000","last_peered":"2024-09-05T13:15:42.664436+0000","last_clean":"2024-09-05T13:15:42.664436+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:42.664436+0000","last_undegraded":"2024-09-05T13:15:42.664436+0000","last_fullsized":"2024-09-05T13:15:42.664436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9032,"log_dups_size":0,"ondisk_log_size":9032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36970496,"num_objects":9026,"num_object_clones":0,"num_object_copies":18052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7042,"num_write_kb":28168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9220","reported_seq":9343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.703100+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:42.703100+0000","last_peered":"2024-09-05T13:15:42.703100+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:42.703100+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9220,"log_dups_size":0,"ondisk_log_size":9220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9216,"num_object_clones":0,"num_object_copies":18432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7031,"num_write_kb":28124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4724}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9123","reported_seq":9186,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:42.630032+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:42.630032+0000","last_peered":"2024-09-05T13:15:42.630032+0000","last_clean":"2024-09-05T13:15:42.630032+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:42.630032+0000","last_undegraded":"2024-09-05T13:15:42.630032+0000","last_fullsized":"2024-09-05T13:15:42.630032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9123,"log_dups_size":0,"ondisk_log_size":9123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9123,"num_write_kb":36492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9108","reported_seq":9238,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:42.072255+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:42.072255+0000","last_peered":"2024-09-05T13:15:42.072255+0000","last_clean":"2024-09-05T13:15:42.072255+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:42.072255+0000","last_undegraded":"2024-09-05T13:15:42.072255+0000","last_fullsized":"2024-09-05T13:15:42.072255+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5648,"num_write_kb":22592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9185","reported_seq":9319,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.192986+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:42.192986+0000","last_peered":"2024-09-05T13:15:42.192986+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:42.192986+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9185,"log_dups_size":0,"ondisk_log_size":9185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5640,"num_write_kb":22560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4729},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8977","reported_seq":9111,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.188819+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:42.188819+0000","last_peered":"2024-09-05T13:15:42.188819+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:42.188819+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36745216,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5296,"num_write_kb":21184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4535},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9213","reported_seq":9340,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:42.189098+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:42.189098+0000","last_peered":"2024-09-05T13:15:42.189098+0000","last_clean":"2024-09-05T13:15:42.189098+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:42.189098+0000","last_undegraded":"2024-09-05T13:15:42.189098+0000","last_fullsized":"2024-09-05T13:15:42.189098+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9213,"log_dups_size":0,"ondisk_log_size":9213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37728256,"num_objects":9211,"num_object_clones":0,"num_object_copies":18422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9108","reported_seq":9238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.643636+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:42.643636+0000","last_peered":"2024-09-05T13:15:42.643636+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:42.643636+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6762,"num_write_kb":27048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4650}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9113","reported_seq":9246,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.104541+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:42.104541+0000","last_peered":"2024-09-05T13:15:42.104541+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:42.104541+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9113,"log_dups_size":0,"ondisk_log_size":9113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37294080,"num_objects":9105,"num_object_clones":0,"num_object_copies":18210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5480,"num_write_kb":21920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4651},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9122","reported_seq":16452,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.684748+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:42.684748+0000","last_peered":"2024-09-05T13:15:42.684748+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:42.684748+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9122,"log_dups_size":0,"ondisk_log_size":9122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6717,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":14774272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":778},{"shards":"2","objects":8337}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8931","reported_seq":9049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.610403+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:42.610403+0000","last_peered":"2024-09-05T13:15:42.610403+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:42.610403+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8931,"log_dups_size":0,"ondisk_log_size":8931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36556800,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6440,"num_write_kb":25760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4538}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9067","reported_seq":9203,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.261200+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:42.261200+0000","last_peered":"2024-09-05T13:15:42.261200+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:42.261200+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9067,"log_dups_size":0,"ondisk_log_size":9067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37130240,"num_objects":9065,"num_object_clones":0,"num_object_copies":18130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5218,"num_write_kb":20872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4646},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9154","reported_seq":9336,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.244550+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:42.244550+0000","last_peered":"2024-09-05T13:15:42.244550+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:42.244550+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9154,"log_dups_size":0,"ondisk_log_size":9154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37462016,"num_objects":9146,"num_object_clones":0,"num_object_copies":18292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5053,"num_write_kb":20212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4702},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9173","reported_seq":9306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.195345+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:42.195345+0000","last_peered":"2024-09-05T13:15:42.195345+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:42.195345+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9173,"log_dups_size":0,"ondisk_log_size":9173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556224,"num_objects":9169,"num_object_clones":0,"num_object_copies":18338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4603},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9261","reported_seq":9396,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.011032+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:42.011032+0000","last_peered":"2024-09-05T13:15:42.011032+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:42.011032+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884387,"num_objects":9250,"num_object_clones":0,"num_object_copies":18500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5929,"num_write_kb":23716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4685},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":596976099,"num_objects":145747,"num_object_clones":0,"num_object_copies":291494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":145747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101546,"num_write_kb":406184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":14786560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1205170176,"data_stored":1205087077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145828,"ondisk_log_size":145828,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686824,"kb_used_data":434392,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93685016,"statfs":{"total":96636764160,"available":95933456384,"internally_reserved":0,"allocated":444817408,"data_stored":444572318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":810500,"kb_used_data":508148,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93561340,"statfs":{"total":96636764160,"available":95806812160,"internally_reserved":0,"allocated":520343552,"data_stored":520094385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,4,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428616,"kb_used_data":234488,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93943224,"statfs":{"total":96636764160,"available":96197861376,"internally_reserved":0,"allocated":240115712,"data_stored":239794285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198766255},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":520531968,"data_stored":520528355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239828992,"data_stored":239753119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444809216,"data_stored":444805603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:45.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 192 20 146957 146937 2.98866 1.75 0.0202339 0.0261327 2024-09-05T13:15:46.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 193 20 147440 147420 2.98295 1.88672 0.0363299 0.0261833 2024-09-05T13:15:47.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 194 20 147916 147896 2.97716 1.85938 0.0384395 0.0262332 2024-09-05T13:15:47.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:48.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 195 20 148364 148344 2.97087 1.75 0.121757 0.0262883 2024-09-05T13:15:48.317 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:48.319 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:48.348 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-05T13:15:47.583680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602784246,"num_objects":147167,"num_object_clones":0,"num_object_copies":294334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":147167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102966,"num_write_kb":411858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3613,"num_bytes_recovered":14786579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147249,"ondisk_log_size":147249,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948272,"kb_used_data":1191040,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281167248,"statfs":{"total":289910292480,"available":287915261952,"internally_reserved":0,"allocated":1219624960,"data_stored":1218809276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":775343110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,7,6,3,10,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24764416,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-304,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6046,"num_write_kb":24184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011171"},"pg_stats":[{"pgid":"2.f","version":"43'9199","reported_seq":9269,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.196462+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:47.196462+0000","last_peered":"2024-09-05T13:15:47.196462+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:47.196462+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9199,"log_dups_size":0,"ondisk_log_size":9199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679104,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9199,"num_write_kb":36796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4778},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9032","reported_seq":9132,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:42.664436+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:42.664436+0000","last_peered":"2024-09-05T13:15:42.664436+0000","last_clean":"2024-09-05T13:15:42.664436+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:42.664436+0000","last_undegraded":"2024-09-05T13:15:42.664436+0000","last_fullsized":"2024-09-05T13:15:42.664436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9032,"log_dups_size":0,"ondisk_log_size":9032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36970496,"num_objects":9026,"num_object_clones":0,"num_object_copies":18052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7042,"num_write_kb":28168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9220","reported_seq":9343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.703100+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:42.703100+0000","last_peered":"2024-09-05T13:15:42.703100+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:42.703100+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9220,"log_dups_size":0,"ondisk_log_size":9220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9216,"num_object_clones":0,"num_object_copies":18432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7031,"num_write_kb":28124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4724}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9123","reported_seq":9186,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:42.630032+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:42.630032+0000","last_peered":"2024-09-05T13:15:42.630032+0000","last_clean":"2024-09-05T13:15:42.630032+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:42.630032+0000","last_undegraded":"2024-09-05T13:15:42.630032+0000","last_fullsized":"2024-09-05T13:15:42.630032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9123,"log_dups_size":0,"ondisk_log_size":9123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9123,"num_write_kb":36492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9250","reported_seq":9378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:47.277373+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:47.277373+0000","last_peered":"2024-09-05T13:15:47.277373+0000","last_clean":"2024-09-05T13:15:47.277373+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:47.277373+0000","last_undegraded":"2024-09-05T13:15:47.277373+0000","last_fullsized":"2024-09-05T13:15:47.277373+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9250,"log_dups_size":0,"ondisk_log_size":9250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37859328,"num_objects":9243,"num_object_clones":0,"num_object_copies":18486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5790,"num_write_kb":23160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9331","reported_seq":9465,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.263115+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:47.263115+0000","last_peered":"2024-09-05T13:15:47.263115+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:47.263115+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9331,"log_dups_size":0,"ondisk_log_size":9331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5786,"num_write_kb":23144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9122","reported_seq":9257,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.234402+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:47.234402+0000","last_peered":"2024-09-05T13:15:47.234402+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:47.234402+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9122,"log_dups_size":0,"ondisk_log_size":9122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37339136,"num_objects":9116,"num_object_clones":0,"num_object_copies":18232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5441,"num_write_kb":21764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4680},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9350","reported_seq":9477,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:47.162280+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:47.162280+0000","last_peered":"2024-09-05T13:15:47.162280+0000","last_clean":"2024-09-05T13:15:47.162280+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:47.162280+0000","last_undegraded":"2024-09-05T13:15:47.162280+0000","last_fullsized":"2024-09-05T13:15:47.162280+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9350,"log_dups_size":0,"ondisk_log_size":9350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5960,"num_write_kb":23840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9108","reported_seq":9238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.643636+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:42.643636+0000","last_peered":"2024-09-05T13:15:42.643636+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:42.643636+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6762,"num_write_kb":27048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4650}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9244","reported_seq":9377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.254031+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:47.254031+0000","last_peered":"2024-09-05T13:15:47.254031+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:47.254031+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9244,"log_dups_size":0,"ondisk_log_size":9244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37830656,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5611,"num_write_kb":22444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4782},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9122","reported_seq":16452,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.684748+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:42.684748+0000","last_peered":"2024-09-05T13:15:42.684748+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:42.684748+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9122,"log_dups_size":0,"ondisk_log_size":9122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6717,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":14774272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":778},{"shards":"2","objects":8337}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8931","reported_seq":9049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.610403+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:42.610403+0000","last_peered":"2024-09-05T13:15:42.610403+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:42.610403+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8931,"log_dups_size":0,"ondisk_log_size":8931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36556800,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6440,"num_write_kb":25760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4538}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9206","reported_seq":9342,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.177699+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:47.177699+0000","last_peered":"2024-09-05T13:15:47.177699+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:47.177699+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9206,"log_dups_size":0,"ondisk_log_size":9206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5357,"num_write_kb":21428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4785},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9302","reported_seq":9485,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.247225+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:47.247225+0000","last_peered":"2024-09-05T13:15:47.247225+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:47.247225+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9302,"log_dups_size":0,"ondisk_log_size":9302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38068224,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5201,"num_write_kb":20804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9303","reported_seq":9437,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.140365+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:47.140365+0000","last_peered":"2024-09-05T13:15:47.140365+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:47.140365+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38088704,"num_objects":9299,"num_object_clones":0,"num_object_copies":18598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5433,"num_write_kb":21732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9403","reported_seq":9536,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.264577+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:47.264577+0000","last_peered":"2024-09-05T13:15:47.264577+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:47.264577+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466019,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":24284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4827},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602784227,"num_objects":147165,"num_object_clones":0,"num_object_copies":294330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":147165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102964,"num_write_kb":411856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":14786560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1219833856,"data_stored":1219750757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147246,"ondisk_log_size":147246,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686824,"kb_used_data":434392,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93685016,"statfs":{"total":96636764160,"available":95933456384,"internally_reserved":0,"allocated":444817408,"data_stored":444572318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":821352,"kb_used_data":514840,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93550488,"statfs":{"total":96636764160,"available":95795699712,"internally_reserved":0,"allocated":527196160,"data_stored":526946993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440096,"kb_used_data":241808,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93931744,"statfs":{"total":96636764160,"available":96186105856,"internally_reserved":0,"allocated":247611392,"data_stored":247289965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,1,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527536128,"data_stored":527532515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":247488512,"data_stored":247412639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444809216,"data_stored":444805603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:48.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:48.921 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:48.946 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-05T13:15:47.583680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602784246,"num_objects":147167,"num_object_clones":0,"num_object_copies":294334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":147167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102966,"num_write_kb":411858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3613,"num_bytes_recovered":14786579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147249,"ondisk_log_size":147249,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948272,"kb_used_data":1191040,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281167248,"statfs":{"total":289910292480,"available":287915261952,"internally_reserved":0,"allocated":1219624960,"data_stored":1218809276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":775343110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,7,6,3,10,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24764416,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-304,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6046,"num_write_kb":24184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011171"},"pg_stats":[{"pgid":"2.f","version":"43'9199","reported_seq":9269,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.196462+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:47.196462+0000","last_peered":"2024-09-05T13:15:47.196462+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:47.196462+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9199,"log_dups_size":0,"ondisk_log_size":9199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679104,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9199,"num_write_kb":36796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4778},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9032","reported_seq":9132,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:42.664436+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:42.664436+0000","last_peered":"2024-09-05T13:15:42.664436+0000","last_clean":"2024-09-05T13:15:42.664436+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:42.664436+0000","last_undegraded":"2024-09-05T13:15:42.664436+0000","last_fullsized":"2024-09-05T13:15:42.664436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9032,"log_dups_size":0,"ondisk_log_size":9032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36970496,"num_objects":9026,"num_object_clones":0,"num_object_copies":18052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7042,"num_write_kb":28168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9220","reported_seq":9343,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.703100+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:42.703100+0000","last_peered":"2024-09-05T13:15:42.703100+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:42.703100+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9220,"log_dups_size":0,"ondisk_log_size":9220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9216,"num_object_clones":0,"num_object_copies":18432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7031,"num_write_kb":28124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4724}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9123","reported_seq":9186,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:42.630032+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:42.630032+0000","last_peered":"2024-09-05T13:15:42.630032+0000","last_clean":"2024-09-05T13:15:42.630032+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:42.630032+0000","last_undegraded":"2024-09-05T13:15:42.630032+0000","last_fullsized":"2024-09-05T13:15:42.630032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9123,"log_dups_size":0,"ondisk_log_size":9123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37367808,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9123,"num_write_kb":36492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9250","reported_seq":9378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:47.277373+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:47.277373+0000","last_peered":"2024-09-05T13:15:47.277373+0000","last_clean":"2024-09-05T13:15:47.277373+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:47.277373+0000","last_undegraded":"2024-09-05T13:15:47.277373+0000","last_fullsized":"2024-09-05T13:15:47.277373+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9250,"log_dups_size":0,"ondisk_log_size":9250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37859328,"num_objects":9243,"num_object_clones":0,"num_object_copies":18486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5790,"num_write_kb":23160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9331","reported_seq":9465,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.263115+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:47.263115+0000","last_peered":"2024-09-05T13:15:47.263115+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:47.263115+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9331,"log_dups_size":0,"ondisk_log_size":9331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5786,"num_write_kb":23144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9122","reported_seq":9257,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.234402+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:47.234402+0000","last_peered":"2024-09-05T13:15:47.234402+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:47.234402+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9122,"log_dups_size":0,"ondisk_log_size":9122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37339136,"num_objects":9116,"num_object_clones":0,"num_object_copies":18232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5441,"num_write_kb":21764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4680},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9350","reported_seq":9477,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:47.162280+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:47.162280+0000","last_peered":"2024-09-05T13:15:47.162280+0000","last_clean":"2024-09-05T13:15:47.162280+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:47.162280+0000","last_undegraded":"2024-09-05T13:15:47.162280+0000","last_fullsized":"2024-09-05T13:15:47.162280+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9350,"log_dups_size":0,"ondisk_log_size":9350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5960,"num_write_kb":23840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9108","reported_seq":9238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.643636+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:42.643636+0000","last_peered":"2024-09-05T13:15:42.643636+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:42.643636+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6762,"num_write_kb":27048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4650}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9244","reported_seq":9377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.254031+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:47.254031+0000","last_peered":"2024-09-05T13:15:47.254031+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:47.254031+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9244,"log_dups_size":0,"ondisk_log_size":9244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37830656,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5611,"num_write_kb":22444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4782},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9122","reported_seq":16452,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.684748+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:42.684748+0000","last_peered":"2024-09-05T13:15:42.684748+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:42.684748+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9122,"log_dups_size":0,"ondisk_log_size":9122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6717,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":14774272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":778},{"shards":"2","objects":8337}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8931","reported_seq":9049,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:42.610403+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:42.610403+0000","last_peered":"2024-09-05T13:15:42.610403+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:42.610403+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":8931,"log_dups_size":0,"ondisk_log_size":8931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36556800,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6440,"num_write_kb":25760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4538}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9206","reported_seq":9342,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.177699+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:47.177699+0000","last_peered":"2024-09-05T13:15:47.177699+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:47.177699+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9206,"log_dups_size":0,"ondisk_log_size":9206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5357,"num_write_kb":21428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4785},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9302","reported_seq":9485,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.247225+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:47.247225+0000","last_peered":"2024-09-05T13:15:47.247225+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:47.247225+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9302,"log_dups_size":0,"ondisk_log_size":9302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38068224,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5201,"num_write_kb":20804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9303","reported_seq":9437,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.140365+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:47.140365+0000","last_peered":"2024-09-05T13:15:47.140365+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:47.140365+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38088704,"num_objects":9299,"num_object_clones":0,"num_object_copies":18598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5433,"num_write_kb":21732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9403","reported_seq":9536,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.264577+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:47.264577+0000","last_peered":"2024-09-05T13:15:47.264577+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:47.264577+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466019,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":24284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4827},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":602784227,"num_objects":147165,"num_object_clones":0,"num_object_copies":294330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":778,"num_objects_unfound":0,"num_objects_dirty":147165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102964,"num_write_kb":411856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3612,"num_bytes_recovered":14786560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1219833856,"data_stored":1219750757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147246,"ondisk_log_size":147246,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":686824,"kb_used_data":434392,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93685016,"statfs":{"total":96636764160,"available":95933456384,"internally_reserved":0,"allocated":444817408,"data_stored":444572318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":821352,"kb_used_data":514840,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93550488,"statfs":{"total":96636764160,"available":95795699712,"internally_reserved":0,"allocated":527196160,"data_stored":526946993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440096,"kb_used_data":241808,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93931744,"statfs":{"total":96636764160,"available":96186105856,"internally_reserved":0,"allocated":247611392,"data_stored":247289965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,1,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527536128,"data_stored":527532515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":247488512,"data_stored":247412639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444809216,"data_stored":444805603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:48.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:48.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'9199", 'reported_seq': 9269, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:47.196462+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:47.196462+0000', 'last_peered': '2024-09-05T13:15:47.196462+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:47.196462+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 9199, 'log_dups_size': 0, 'ondisk_log_size': 9199, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37679104, 'num_objects': 9199, 'num_object_clones': 0, 'num_object_copies': 18398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9199, 'num_write_kb': 36796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4778}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:48.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:48.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'9220", 'reported_seq': 9343, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:42.703100+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:42.703100+0000', 'last_peered': '2024-09-05T13:15:42.703100+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:42.703100+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 9220, 'log_dups_size': 0, 'ondisk_log_size': 9220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37748736, 'num_objects': 9216, 'num_object_clones': 0, 'num_object_copies': 18432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7031, 'num_write_kb': 28124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 4724}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'9331", 'reported_seq': 9465, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:47.263115+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:47.263115+0000', 'last_peered': '2024-09-05T13:15:47.263115+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:47.263115+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 9331, 'log_dups_size': 0, 'ondisk_log_size': 9331, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38199296, 'num_objects': 9326, 'num_object_clones': 0, 'num_object_copies': 18652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5786, 'num_write_kb': 23144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4875}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'9122", 'reported_seq': 9257, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:47.234402+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:47.234402+0000', 'last_peered': '2024-09-05T13:15:47.234402+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:47.234402+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 9122, 'log_dups_size': 0, 'ondisk_log_size': 9122, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37339136, 'num_objects': 9116, 'num_object_clones': 0, 'num_object_copies': 18232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9116, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5441, 'num_write_kb': 21764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4680}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'9108", 'reported_seq': 9238, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:42.643636+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:42.643636+0000', 'last_peered': '2024-09-05T13:15:42.643636+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:42.643636+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 9108, 'log_dups_size': 0, 'ondisk_log_size': 9108, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37277696, 'num_objects': 9101, 'num_object_clones': 0, 'num_object_copies': 18202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6762, 'num_write_kb': 27048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 4650}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'9244", 'reported_seq': 9377, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:47.254031+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:47.254031+0000', 'last_peered': '2024-09-05T13:15:47.254031+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:47.254031+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 9244, 'log_dups_size': 0, 'ondisk_log_size': 9244, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37830656, 'num_objects': 9236, 'num_object_clones': 0, 'num_object_copies': 18472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5611, 'num_write_kb': 22444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4782}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:48.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'9122", 'reported_seq': 16452, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:42.684748+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:42.684748+0000', 'last_peered': '2024-09-05T13:15:42.684748+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:42.684748+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 9122, 'log_dups_size': 0, 'ondisk_log_size': 9122, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37343232, 'num_objects': 9117, 'num_object_clones': 0, 'num_object_copies': 18234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 778, 'num_objects_unfound': 0, 'num_objects_dirty': 9117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6717, 'num_write_kb': 26868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3609, 'num_bytes_recovered': 14774272, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 778}, {'shards': '2', 'objects': 8337}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8931", 'reported_seq': 9049, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:42.610403+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:42.610403+0000', 'last_peered': '2024-09-05T13:15:42.610403+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:42.610403+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 8931, 'log_dups_size': 0, 'ondisk_log_size': 8931, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36556800, 'num_objects': 8925, 'num_object_clones': 0, 'num_object_copies': 17850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8925, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6440, 'num_write_kb': 25760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 4538}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'9206", 'reported_seq': 9342, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:47.177699+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:47.177699+0000', 'last_peered': '2024-09-05T13:15:47.177699+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:47.177699+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 9206, 'log_dups_size': 0, 'ondisk_log_size': 9206, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37699584, 'num_objects': 9204, 'num_object_clones': 0, 'num_object_copies': 18408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9204, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5357, 'num_write_kb': 21428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4785}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'9302", 'reported_seq': 9485, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:47.247225+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:47.247225+0000', 'last_peered': '2024-09-05T13:15:47.247225+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:47.247225+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 9302, 'log_dups_size': 0, 'ondisk_log_size': 9302, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38068224, 'num_objects': 9294, 'num_object_clones': 0, 'num_object_copies': 18588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5201, 'num_write_kb': 20804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4850}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'9303", 'reported_seq': 9437, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:47.140365+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:47.140365+0000', 'last_peered': '2024-09-05T13:15:47.140365+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:47.140365+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 9303, 'log_dups_size': 0, 'ondisk_log_size': 9303, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38088704, 'num_objects': 9299, 'num_object_clones': 0, 'num_object_copies': 18598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5433, 'num_write_kb': 21732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4733}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:48.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'9403", 'reported_seq': 9536, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:47.264577+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:47.264577+0000', 'last_peered': '2024-09-05T13:15:47.264577+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:47.264577+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 9403, 'log_dups_size': 0, 'ondisk_log_size': 9403, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38466019, 'num_objects': 9392, 'num_object_clones': 0, 'num_object_copies': 18784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6071, 'num_write_kb': 24284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4827}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:48.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:49.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 196 20 148780 148760 2.964 1.625 0.0334578 0.0263496 2024-09-05T13:15:49.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:49.622 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/294334 objects degraded (16.684%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":276,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147167,"data_bytes":602784246,"bytes_used":1995030528,"bytes_avail":287915261952,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":294334,"degraded_ratio":0.16684446920845025,"misplaced_objects":778,"misplaced_total":294334,"misplaced_ratio":0.0026432556211650709,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103668,"recovering_keys_per_sec":0,"num_objects_recovered":304,"num_bytes_recovered":1245184,"num_keys_recovered":0,"write_bytes_sec":2061781,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:49.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:49.645 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:50.173 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:50.202 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-05T13:15:49.584377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":606339574,"num_objects":148035,"num_object_clones":0,"num_object_copies":296070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":629,"num_objects_unfound":0,"num_objects_dirty":148035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103834,"num_write_kb":415330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3762,"num_bytes_recovered":15396883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148117,"ondisk_log_size":148117,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1957020,"kb_used_data":1195628,"kb_used_omap":12,"kb_used_meta":761331,"kb_avail":281158500,"statfs":{"total":289910292480,"available":287906304000,"internally_reserved":0,"allocated":1224323072,"data_stored":1223507388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":779602950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,7,1,12,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22450176,"num_objects":5481,"num_object_clones":0,"num_object_copies":10962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-453,"num_objects_unfound":0,"num_objects_dirty":5481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5481,"num_write_kb":21924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":1855488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011223"},"pg_stats":[{"pgid":"2.f","version":"43'9199","reported_seq":9269,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.196462+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:47.196462+0000","last_peered":"2024-09-05T13:15:47.196462+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:47.196462+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9199,"log_dups_size":0,"ondisk_log_size":9199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37679104,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9199,"num_write_kb":36796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4778},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9171","reported_seq":9271,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:47.709299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:47.709299+0000","last_peered":"2024-09-05T13:15:47.709299+0000","last_clean":"2024-09-05T13:15:47.709299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:47.709299+0000","last_undegraded":"2024-09-05T13:15:47.709299+0000","last_fullsized":"2024-09-05T13:15:47.709299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9171,"log_dups_size":0,"ondisk_log_size":9171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7181,"num_write_kb":28724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9362","reported_seq":9485,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.709560+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:47.709560+0000","last_peered":"2024-09-05T13:15:47.709560+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:47.709560+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9362,"log_dups_size":0,"ondisk_log_size":9362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38330368,"num_objects":9358,"num_object_clones":0,"num_object_copies":18716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7173,"num_write_kb":28692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":4866}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9281","reported_seq":9346,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:47.712731+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:47.712731+0000","last_peered":"2024-09-05T13:15:47.712731+0000","last_clean":"2024-09-05T13:15:47.712731+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:47.712731+0000","last_undegraded":"2024-09-05T13:15:47.712731+0000","last_fullsized":"2024-09-05T13:15:47.712731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9281,"log_dups_size":0,"ondisk_log_size":9281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38014976,"num_objects":9281,"num_object_clones":0,"num_object_copies":18562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9281,"num_write_kb":37124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9250","reported_seq":9378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:47.277373+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:47.277373+0000","last_peered":"2024-09-05T13:15:47.277373+0000","last_clean":"2024-09-05T13:15:47.277373+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:47.277373+0000","last_undegraded":"2024-09-05T13:15:47.277373+0000","last_fullsized":"2024-09-05T13:15:47.277373+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9250,"log_dups_size":0,"ondisk_log_size":9250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37859328,"num_objects":9243,"num_object_clones":0,"num_object_copies":18486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5790,"num_write_kb":23160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9331","reported_seq":9465,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.263115+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:47.263115+0000","last_peered":"2024-09-05T13:15:47.263115+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:47.263115+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9331,"log_dups_size":0,"ondisk_log_size":9331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5786,"num_write_kb":23144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9122","reported_seq":9257,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.234402+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:47.234402+0000","last_peered":"2024-09-05T13:15:47.234402+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:47.234402+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9122,"log_dups_size":0,"ondisk_log_size":9122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37339136,"num_objects":9116,"num_object_clones":0,"num_object_copies":18232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5441,"num_write_kb":21764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4680},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9350","reported_seq":9477,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:47.162280+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:47.162280+0000","last_peered":"2024-09-05T13:15:47.162280+0000","last_clean":"2024-09-05T13:15:47.162280+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:47.162280+0000","last_undegraded":"2024-09-05T13:15:47.162280+0000","last_fullsized":"2024-09-05T13:15:47.162280+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9350,"log_dups_size":0,"ondisk_log_size":9350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5960,"num_write_kb":23840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9252","reported_seq":9382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.703596+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:47.703596+0000","last_peered":"2024-09-05T13:15:47.703596+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:47.703596+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9252,"log_dups_size":0,"ondisk_log_size":9252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37867520,"num_objects":9245,"num_object_clones":0,"num_object_copies":18490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6906,"num_write_kb":27624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4794}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9244","reported_seq":9377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.254031+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:47.254031+0000","last_peered":"2024-09-05T13:15:47.254031+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:47.254031+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9244,"log_dups_size":0,"ondisk_log_size":9244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37830656,"num_objects":9236,"num_object_clones":0,"num_object_copies":18472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5611,"num_write_kb":22444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4782},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9266","reported_seq":16894,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.709260+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:47.709260+0000","last_peered":"2024-09-05T13:15:47.709260+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:47.709260+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9266,"log_dups_size":0,"ondisk_log_size":9266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933056,"num_objects":9261,"num_object_clones":0,"num_object_copies":18522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":629,"num_objects_unfound":0,"num_objects_dirty":9261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6861,"num_write_kb":27444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3758,"num_bytes_recovered":15384576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":629},{"shards":"2","objects":8630}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9072","reported_seq":9190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.689517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:47.689517+0000","last_peered":"2024-09-05T13:15:47.689517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:47.689517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9072,"log_dups_size":0,"ondisk_log_size":9072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37134336,"num_objects":9066,"num_object_clones":0,"num_object_copies":18132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6581,"num_write_kb":26324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4679}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9206","reported_seq":9342,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.177699+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:47.177699+0000","last_peered":"2024-09-05T13:15:47.177699+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:47.177699+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9206,"log_dups_size":0,"ondisk_log_size":9206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37699584,"num_objects":9204,"num_object_clones":0,"num_object_copies":18408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5357,"num_write_kb":21428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4785},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9302","reported_seq":9485,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.247225+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:47.247225+0000","last_peered":"2024-09-05T13:15:47.247225+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:47.247225+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9302,"log_dups_size":0,"ondisk_log_size":9302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38068224,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5201,"num_write_kb":20804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9303","reported_seq":9437,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.140365+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:47.140365+0000","last_peered":"2024-09-05T13:15:47.140365+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:47.140365+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38088704,"num_objects":9299,"num_object_clones":0,"num_object_copies":18598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5433,"num_write_kb":21732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9403","reported_seq":9536,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:47.264577+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:47.264577+0000","last_peered":"2024-09-05T13:15:47.264577+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:47.264577+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466019,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":24284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4827},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":606339555,"num_objects":148033,"num_object_clones":0,"num_object_copies":296066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":629,"num_objects_unfound":0,"num_objects_dirty":148033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103832,"num_write_kb":415328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3761,"num_bytes_recovered":15396864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1224536064,"data_stored":1224452965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148114,"ondisk_log_size":148114,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695572,"kb_used_data":438980,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93676268,"statfs":{"total":96636764160,"available":95924498432,"internally_reserved":0,"allocated":449515520,"data_stored":449270430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":821352,"kb_used_data":514840,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93550488,"statfs":{"total":96636764160,"available":95795699712,"internally_reserved":0,"allocated":527196160,"data_stored":526946993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,0,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440096,"kb_used_data":241808,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93931744,"statfs":{"total":96636764160,"available":96186105856,"internally_reserved":0,"allocated":247611392,"data_stored":247289965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,1,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527536128,"data_stored":527532515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":247488512,"data_stored":247412639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":449511424,"data_stored":449507811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:50.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 197 20 149199 149179 2.95721 1.63672 0.0823546 0.0264104 2024-09-05T13:15:51.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 198 20 149672 149652 2.95161 1.84766 0.0377885 0.0264592 2024-09-05T13:15:52.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 199 20 150144 150124 2.94604 1.84375 0.024715 0.0265106 2024-09-05T13:15:53.204 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:53.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:15:53.239115+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0265622 2024-09-05T13:15:53.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:15:53.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 200 20 150609 150589 2.9404 1.81641 0.0724893 0.0265622 2024-09-05T13:15:53.749 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:53.770 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-05T13:15:53.585883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615375350,"num_objects":150241,"num_object_clones":0,"num_object_copies":300482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":150241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106040,"num_write_kb":424154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3931,"num_bytes_recovered":16089107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150323,"ondisk_log_size":150323,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1971452,"kb_used_data":1214220,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281144068,"statfs":{"total":289910292480,"available":287891525632,"internally_reserved":0,"allocated":1243361280,"data_stored":1242545596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":775343110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,6,4,14,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-471,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6791,"num_write_kb":27164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":1929216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011090"},"pg_stats":[{"pgid":"2.f","version":"43'9335","reported_seq":9405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.192777+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:52.192777+0000","last_peered":"2024-09-05T13:15:52.192777+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:52.192777+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9335,"log_dups_size":0,"ondisk_log_size":9335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38236160,"num_objects":9335,"num_object_clones":0,"num_object_copies":18670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9335,"num_write_kb":37340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4914},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9314","reported_seq":9414,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.619152+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:52.619152+0000","last_peered":"2024-09-05T13:15:52.619152+0000","last_clean":"2024-09-05T13:15:52.619152+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:52.619152+0000","last_undegraded":"2024-09-05T13:15:52.619152+0000","last_fullsized":"2024-09-05T13:15:52.619152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9314,"log_dups_size":0,"ondisk_log_size":9314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38125568,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7324,"num_write_kb":29296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9502","reported_seq":9625,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.726635+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:52.726635+0000","last_peered":"2024-09-05T13:15:52.726635+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:52.726635+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38903808,"num_objects":9498,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7313,"num_write_kb":29252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5006}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9433","reported_seq":9498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.714979+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:52.714979+0000","last_peered":"2024-09-05T13:15:52.714979+0000","last_clean":"2024-09-05T13:15:52.714979+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:52.714979+0000","last_undegraded":"2024-09-05T13:15:52.714979+0000","last_fullsized":"2024-09-05T13:15:52.714979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38637568,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9433,"num_write_kb":37732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9376","reported_seq":9506,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.233058+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:52.233058+0000","last_peered":"2024-09-05T13:15:52.233058+0000","last_clean":"2024-09-05T13:15:52.233058+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:52.233058+0000","last_undegraded":"2024-09-05T13:15:52.233058+0000","last_fullsized":"2024-09-05T13:15:52.233058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9376,"log_dups_size":0,"ondisk_log_size":9376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38375424,"num_objects":9369,"num_object_clones":0,"num_object_copies":18738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5916,"num_write_kb":23664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9470","reported_seq":9601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.228383+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:52.228383+0000","last_peered":"2024-09-05T13:15:52.228383+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:52.228383+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5925,"num_write_kb":23700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5014},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9253","reported_seq":9388,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.259393+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:52.259393+0000","last_peered":"2024-09-05T13:15:52.259393+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:52.259393+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9253,"log_dups_size":0,"ondisk_log_size":9253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37875712,"num_objects":9247,"num_object_clones":0,"num_object_copies":18494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5572,"num_write_kb":22288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4811},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9490","reported_seq":9617,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.299058+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:52.299058+0000","last_peered":"2024-09-05T13:15:52.299058+0000","last_clean":"2024-09-05T13:15:52.299058+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:52.299058+0000","last_undegraded":"2024-09-05T13:15:52.299058+0000","last_fullsized":"2024-09-05T13:15:52.299058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38862848,"num_objects":9488,"num_object_clones":0,"num_object_copies":18976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6100,"num_write_kb":24400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9404","reported_seq":9534,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.648049+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:52.648049+0000","last_peered":"2024-09-05T13:15:52.648049+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:52.648049+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38490112,"num_objects":9397,"num_object_clones":0,"num_object_copies":18794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4946}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9381","reported_seq":9513,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.246736+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:52.246736+0000","last_peered":"2024-09-05T13:15:52.246736+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:52.246736+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5748,"num_write_kb":22992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9389","reported_seq":17355,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.735727+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:52.735727+0000","last_peered":"2024-09-05T13:15:52.735727+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:52.735727+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9389,"log_dups_size":0,"ondisk_log_size":9389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38436864,"num_objects":9384,"num_object_clones":0,"num_object_copies":18768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":9384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6984,"num_write_kb":27936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":460},{"shards":"2","objects":8922}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9212","reported_seq":9330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.672883+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:52.672883+0000","last_peered":"2024-09-05T13:15:52.672883+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:52.672883+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9212,"log_dups_size":0,"ondisk_log_size":9212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37707776,"num_objects":9206,"num_object_clones":0,"num_object_copies":18412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6721,"num_write_kb":26884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4819}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9356","reported_seq":9490,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.290346+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:52.290346+0000","last_peered":"2024-09-05T13:15:52.290346+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:52.290346+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9356,"log_dups_size":0,"ondisk_log_size":9356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38313984,"num_objects":9354,"num_object_clones":0,"num_object_copies":18708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4935},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9426","reported_seq":9609,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.209524+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:52.209524+0000","last_peered":"2024-09-05T13:15:52.209524+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:52.209524+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9426,"log_dups_size":0,"ondisk_log_size":9426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38576128,"num_objects":9418,"num_object_clones":0,"num_object_copies":18836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5325,"num_write_kb":21300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4974},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9460","reported_seq":9594,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.248992+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:52.248992+0000","last_peered":"2024-09-05T13:15:52.248992+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:52.248992+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9460,"log_dups_size":0,"ondisk_log_size":9460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5590,"num_write_kb":22360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4890},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9519","reported_seq":9654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.266889+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:52.266889+0000","last_peered":"2024-09-05T13:15:52.266889+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:52.266889+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9519,"log_dups_size":0,"ondisk_log_size":9519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38941155,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6187,"num_write_kb":24748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4943},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":615375331,"num_objects":150239,"num_object_clones":0,"num_object_copies":300478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":150239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106038,"num_write_kb":424152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3930,"num_bytes_recovered":16089088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1243471872,"data_stored":1243388773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150320,"ondisk_log_size":150320,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687680,"kb_used_data":443568,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93684160,"statfs":{"total":96636764160,"available":95932579840,"internally_reserved":0,"allocated":454213632,"data_stored":453968542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":832300,"kb_used_data":521628,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93539540,"statfs":{"total":96636764160,"available":95784488960,"internally_reserved":0,"allocated":534147072,"data_stored":533897905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,1,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451472,"kb_used_data":249024,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93920368,"statfs":{"total":96636764160,"available":96174456832,"internally_reserved":0,"allocated":255000576,"data_stored":254679149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207285935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,4,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":534339584,"data_stored":534335971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255033344,"data_stored":254957471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454098944,"data_stored":454095331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:53.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:54.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 201 20 151018 150998 2.93372 1.59766 0.0361687 0.0266223 2024-09-05T13:15:54.375 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:54.376 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:54.399 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-05T13:15:53.585883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615375350,"num_objects":150241,"num_object_clones":0,"num_object_copies":300482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":150241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106040,"num_write_kb":424154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3931,"num_bytes_recovered":16089107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150323,"ondisk_log_size":150323,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1971452,"kb_used_data":1214220,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281144068,"statfs":{"total":289910292480,"available":287891525632,"internally_reserved":0,"allocated":1243361280,"data_stored":1242545596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":775343110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,6,4,14,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-471,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6791,"num_write_kb":27164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":471,"num_bytes_recovered":1929216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011090"},"pg_stats":[{"pgid":"2.f","version":"43'9335","reported_seq":9405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.192777+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:52.192777+0000","last_peered":"2024-09-05T13:15:52.192777+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:52.192777+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9335,"log_dups_size":0,"ondisk_log_size":9335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38236160,"num_objects":9335,"num_object_clones":0,"num_object_copies":18670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9335,"num_write_kb":37340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4914},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9314","reported_seq":9414,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.619152+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:52.619152+0000","last_peered":"2024-09-05T13:15:52.619152+0000","last_clean":"2024-09-05T13:15:52.619152+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:52.619152+0000","last_undegraded":"2024-09-05T13:15:52.619152+0000","last_fullsized":"2024-09-05T13:15:52.619152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9314,"log_dups_size":0,"ondisk_log_size":9314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38125568,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7324,"num_write_kb":29296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9502","reported_seq":9625,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.726635+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:52.726635+0000","last_peered":"2024-09-05T13:15:52.726635+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:52.726635+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38903808,"num_objects":9498,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7313,"num_write_kb":29252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5006}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9433","reported_seq":9498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.714979+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:52.714979+0000","last_peered":"2024-09-05T13:15:52.714979+0000","last_clean":"2024-09-05T13:15:52.714979+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:52.714979+0000","last_undegraded":"2024-09-05T13:15:52.714979+0000","last_fullsized":"2024-09-05T13:15:52.714979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38637568,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9433,"num_write_kb":37732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9376","reported_seq":9506,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.233058+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:52.233058+0000","last_peered":"2024-09-05T13:15:52.233058+0000","last_clean":"2024-09-05T13:15:52.233058+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:52.233058+0000","last_undegraded":"2024-09-05T13:15:52.233058+0000","last_fullsized":"2024-09-05T13:15:52.233058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9376,"log_dups_size":0,"ondisk_log_size":9376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38375424,"num_objects":9369,"num_object_clones":0,"num_object_copies":18738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5916,"num_write_kb":23664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9470","reported_seq":9601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.228383+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:52.228383+0000","last_peered":"2024-09-05T13:15:52.228383+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:52.228383+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5925,"num_write_kb":23700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5014},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9253","reported_seq":9388,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.259393+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:52.259393+0000","last_peered":"2024-09-05T13:15:52.259393+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:52.259393+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9253,"log_dups_size":0,"ondisk_log_size":9253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37875712,"num_objects":9247,"num_object_clones":0,"num_object_copies":18494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5572,"num_write_kb":22288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4811},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9490","reported_seq":9617,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.299058+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:52.299058+0000","last_peered":"2024-09-05T13:15:52.299058+0000","last_clean":"2024-09-05T13:15:52.299058+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:52.299058+0000","last_undegraded":"2024-09-05T13:15:52.299058+0000","last_fullsized":"2024-09-05T13:15:52.299058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38862848,"num_objects":9488,"num_object_clones":0,"num_object_copies":18976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6100,"num_write_kb":24400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9404","reported_seq":9534,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.648049+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:52.648049+0000","last_peered":"2024-09-05T13:15:52.648049+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:52.648049+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38490112,"num_objects":9397,"num_object_clones":0,"num_object_copies":18794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4946}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9381","reported_seq":9513,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.246736+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:52.246736+0000","last_peered":"2024-09-05T13:15:52.246736+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:52.246736+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5748,"num_write_kb":22992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9389","reported_seq":17355,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.735727+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:52.735727+0000","last_peered":"2024-09-05T13:15:52.735727+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:52.735727+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9389,"log_dups_size":0,"ondisk_log_size":9389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38436864,"num_objects":9384,"num_object_clones":0,"num_object_copies":18768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":9384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6984,"num_write_kb":27936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":460},{"shards":"2","objects":8922}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9212","reported_seq":9330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.672883+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:52.672883+0000","last_peered":"2024-09-05T13:15:52.672883+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:52.672883+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9212,"log_dups_size":0,"ondisk_log_size":9212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37707776,"num_objects":9206,"num_object_clones":0,"num_object_copies":18412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6721,"num_write_kb":26884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4819}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9356","reported_seq":9490,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.290346+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:52.290346+0000","last_peered":"2024-09-05T13:15:52.290346+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:52.290346+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9356,"log_dups_size":0,"ondisk_log_size":9356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38313984,"num_objects":9354,"num_object_clones":0,"num_object_copies":18708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4935},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9426","reported_seq":9609,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.209524+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:52.209524+0000","last_peered":"2024-09-05T13:15:52.209524+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:52.209524+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9426,"log_dups_size":0,"ondisk_log_size":9426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38576128,"num_objects":9418,"num_object_clones":0,"num_object_copies":18836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5325,"num_write_kb":21300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4974},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9460","reported_seq":9594,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.248992+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:52.248992+0000","last_peered":"2024-09-05T13:15:52.248992+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:52.248992+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9460,"log_dups_size":0,"ondisk_log_size":9460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5590,"num_write_kb":22360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4890},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9519","reported_seq":9654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.266889+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:52.266889+0000","last_peered":"2024-09-05T13:15:52.266889+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:52.266889+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9519,"log_dups_size":0,"ondisk_log_size":9519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38941155,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6187,"num_write_kb":24748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4943},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":615375331,"num_objects":150239,"num_object_clones":0,"num_object_copies":300478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":150239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106038,"num_write_kb":424152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3930,"num_bytes_recovered":16089088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1243471872,"data_stored":1243388773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150320,"ondisk_log_size":150320,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687680,"kb_used_data":443568,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93684160,"statfs":{"total":96636764160,"available":95932579840,"internally_reserved":0,"allocated":454213632,"data_stored":453968542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":832300,"kb_used_data":521628,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93539540,"statfs":{"total":96636764160,"available":95784488960,"internally_reserved":0,"allocated":534147072,"data_stored":533897905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,1,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451472,"kb_used_data":249024,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93920368,"statfs":{"total":96636764160,"available":96174456832,"internally_reserved":0,"allocated":255000576,"data_stored":254679149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207285935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,4,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":534339584,"data_stored":534335971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255033344,"data_stored":254957471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454098944,"data_stored":454095331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:54.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:54.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'9335", 'reported_seq': 9405, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.192777+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:52.192777+0000', 'last_peered': '2024-09-05T13:15:52.192777+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:52.192777+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 9335, 'log_dups_size': 0, 'ondisk_log_size': 9335, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38236160, 'num_objects': 9335, 'num_object_clones': 0, 'num_object_copies': 18670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9335, 'num_write_kb': 37340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4914}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:54.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:54.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'9502", 'reported_seq': 9625, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.726635+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:52.726635+0000', 'last_peered': '2024-09-05T13:15:52.726635+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:52.726635+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 9502, 'log_dups_size': 0, 'ondisk_log_size': 9502, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38903808, 'num_objects': 9498, 'num_object_clones': 0, 'num_object_copies': 18996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7313, 'num_write_kb': 29252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 5006}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:54.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:54.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'9470", 'reported_seq': 9601, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.228383+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:52.228383+0000', 'last_peered': '2024-09-05T13:15:52.228383+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:52.228383+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 9470, 'log_dups_size': 0, 'ondisk_log_size': 9470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38768640, 'num_objects': 9465, 'num_object_clones': 0, 'num_object_copies': 18930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5925, 'num_write_kb': 23700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5014}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:54.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:54.401 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'9253", 'reported_seq': 9388, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.259393+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:52.259393+0000', 'last_peered': '2024-09-05T13:15:52.259393+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:52.259393+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 9253, 'log_dups_size': 0, 'ondisk_log_size': 9253, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37875712, 'num_objects': 9247, 'num_object_clones': 0, 'num_object_copies': 18494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5572, 'num_write_kb': 22288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4811}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'9404", 'reported_seq': 9534, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.648049+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:52.648049+0000', 'last_peered': '2024-09-05T13:15:52.648049+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:52.648049+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 9404, 'log_dups_size': 0, 'ondisk_log_size': 9404, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38490112, 'num_objects': 9397, 'num_object_clones': 0, 'num_object_copies': 18794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9397, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7058, 'num_write_kb': 28232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 4946}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'9381", 'reported_seq': 9513, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.246736+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:52.246736+0000', 'last_peered': '2024-09-05T13:15:52.246736+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:52.246736+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 9381, 'log_dups_size': 0, 'ondisk_log_size': 9381, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38391808, 'num_objects': 9373, 'num_object_clones': 0, 'num_object_copies': 18746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9373, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5748, 'num_write_kb': 22992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4919}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'9389", 'reported_seq': 17355, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.735727+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:52.735727+0000', 'last_peered': '2024-09-05T13:15:52.735727+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:52.735727+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 9389, 'log_dups_size': 0, 'ondisk_log_size': 9389, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38436864, 'num_objects': 9384, 'num_object_clones': 0, 'num_object_copies': 18768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 460, 'num_objects_unfound': 0, 'num_objects_dirty': 9384, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6984, 'num_write_kb': 27936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3927, 'num_bytes_recovered': 16076800, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 460}, {'shards': '2', 'objects': 8922}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'9212", 'reported_seq': 9330, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.672883+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:52.672883+0000', 'last_peered': '2024-09-05T13:15:52.672883+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:52.672883+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 9212, 'log_dups_size': 0, 'ondisk_log_size': 9212, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37707776, 'num_objects': 9206, 'num_object_clones': 0, 'num_object_copies': 18412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9206, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6721, 'num_write_kb': 26884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 4819}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:54.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'9356", 'reported_seq': 9490, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.290346+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:52.290346+0000', 'last_peered': '2024-09-05T13:15:52.290346+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:52.290346+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 9356, 'log_dups_size': 0, 'ondisk_log_size': 9356, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38313984, 'num_objects': 9354, 'num_object_clones': 0, 'num_object_copies': 18708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5507, 'num_write_kb': 22028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4935}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:54.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:54.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'9426", 'reported_seq': 9609, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.209524+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:52.209524+0000', 'last_peered': '2024-09-05T13:15:52.209524+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:52.209524+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 9426, 'log_dups_size': 0, 'ondisk_log_size': 9426, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38576128, 'num_objects': 9418, 'num_object_clones': 0, 'num_object_copies': 18836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9418, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5325, 'num_write_kb': 21300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4974}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:54.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:54.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'9460", 'reported_seq': 9594, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.248992+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:52.248992+0000', 'last_peered': '2024-09-05T13:15:52.248992+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:52.248992+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 9460, 'log_dups_size': 0, 'ondisk_log_size': 9460, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38731776, 'num_objects': 9456, 'num_object_clones': 0, 'num_object_copies': 18912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9456, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5590, 'num_write_kb': 22360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4890}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:54.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:54.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'9519", 'reported_seq': 9654, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:52.266889+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:52.266889+0000', 'last_peered': '2024-09-05T13:15:52.266889+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:52.266889+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 9519, 'log_dups_size': 0, 'ondisk_log_size': 9519, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38941155, 'num_objects': 9508, 'num_object_clones': 0, 'num_object_copies': 19016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6187, 'num_write_kb': 24748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4943}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:54.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:15:55.100 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:55.100 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/300482 objects degraded (16.343%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":281,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":150241,"data_bytes":615375350,"bytes_used":2018766848,"bytes_avail":287891525632,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":300482,"degraded_ratio":0.16343075458762921,"misplaced_objects":460,"misplaced_total":300482,"misplaced_ratio":0.001530873729541204,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":160619,"recovering_keys_per_sec":0,"num_objects_recovered":471,"num_bytes_recovered":1929216,"num_keys_recovered":0,"write_bytes_sec":2315854,"read_op_per_sec":0,"write_op_per_sec":565},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:15:55.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:15:55.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:55.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 202 20 151429 151409 2.92714 1.60547 0.0462695 0.0266807 2024-09-05T13:15:55.696 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:55.727 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-05T13:15:55.586692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":615375350,"num_objects":150241,"num_object_clones":0,"num_object_copies":300482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":150241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106040,"num_write_kb":424154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3931,"num_bytes_recovered":16089107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150323,"ondisk_log_size":150323,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1988720,"kb_used_data":1221760,"kb_used_omap":12,"kb_used_meta":766899,"kb_avail":281126800,"statfs":{"total":289910292480,"available":287873843200,"internally_reserved":0,"allocated":1251082240,"data_stored":1250266556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":785304582},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,6,5,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18399232,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4492,"num_write_kb":17968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007807"},"pg_stats":[{"pgid":"2.f","version":"43'9335","reported_seq":9405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.192777+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:52.192777+0000","last_peered":"2024-09-05T13:15:52.192777+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:52.192777+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9335,"log_dups_size":0,"ondisk_log_size":9335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38236160,"num_objects":9335,"num_object_clones":0,"num_object_copies":18670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9335,"num_write_kb":37340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4914},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9314","reported_seq":9414,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.619152+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:52.619152+0000","last_peered":"2024-09-05T13:15:52.619152+0000","last_clean":"2024-09-05T13:15:52.619152+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:52.619152+0000","last_undegraded":"2024-09-05T13:15:52.619152+0000","last_fullsized":"2024-09-05T13:15:52.619152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9314,"log_dups_size":0,"ondisk_log_size":9314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38125568,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7324,"num_write_kb":29296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9502","reported_seq":9625,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.726635+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:52.726635+0000","last_peered":"2024-09-05T13:15:52.726635+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:52.726635+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38903808,"num_objects":9498,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7313,"num_write_kb":29252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5006}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9433","reported_seq":9498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.714979+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:52.714979+0000","last_peered":"2024-09-05T13:15:52.714979+0000","last_clean":"2024-09-05T13:15:52.714979+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:52.714979+0000","last_undegraded":"2024-09-05T13:15:52.714979+0000","last_fullsized":"2024-09-05T13:15:52.714979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38637568,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9433,"num_write_kb":37732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9376","reported_seq":9506,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.233058+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:52.233058+0000","last_peered":"2024-09-05T13:15:52.233058+0000","last_clean":"2024-09-05T13:15:52.233058+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:52.233058+0000","last_undegraded":"2024-09-05T13:15:52.233058+0000","last_fullsized":"2024-09-05T13:15:52.233058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9376,"log_dups_size":0,"ondisk_log_size":9376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38375424,"num_objects":9369,"num_object_clones":0,"num_object_copies":18738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5916,"num_write_kb":23664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9470","reported_seq":9601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.228383+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:52.228383+0000","last_peered":"2024-09-05T13:15:52.228383+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:52.228383+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5925,"num_write_kb":23700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5014},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9253","reported_seq":9388,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.259393+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:52.259393+0000","last_peered":"2024-09-05T13:15:52.259393+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:52.259393+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9253,"log_dups_size":0,"ondisk_log_size":9253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37875712,"num_objects":9247,"num_object_clones":0,"num_object_copies":18494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5572,"num_write_kb":22288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4811},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9490","reported_seq":9617,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.299058+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:52.299058+0000","last_peered":"2024-09-05T13:15:52.299058+0000","last_clean":"2024-09-05T13:15:52.299058+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:52.299058+0000","last_undegraded":"2024-09-05T13:15:52.299058+0000","last_fullsized":"2024-09-05T13:15:52.299058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38862848,"num_objects":9488,"num_object_clones":0,"num_object_copies":18976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6100,"num_write_kb":24400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9404","reported_seq":9534,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.648049+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:52.648049+0000","last_peered":"2024-09-05T13:15:52.648049+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:52.648049+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38490112,"num_objects":9397,"num_object_clones":0,"num_object_copies":18794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4946}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9381","reported_seq":9513,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.246736+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:52.246736+0000","last_peered":"2024-09-05T13:15:52.246736+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:52.246736+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38391808,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5748,"num_write_kb":22992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9389","reported_seq":17355,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.735727+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:52.735727+0000","last_peered":"2024-09-05T13:15:52.735727+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:52.735727+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9389,"log_dups_size":0,"ondisk_log_size":9389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38436864,"num_objects":9384,"num_object_clones":0,"num_object_copies":18768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":9384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6984,"num_write_kb":27936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":460},{"shards":"2","objects":8922}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9212","reported_seq":9330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.672883+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:52.672883+0000","last_peered":"2024-09-05T13:15:52.672883+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:52.672883+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9212,"log_dups_size":0,"ondisk_log_size":9212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37707776,"num_objects":9206,"num_object_clones":0,"num_object_copies":18412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6721,"num_write_kb":26884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4819}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9356","reported_seq":9490,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.290346+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:52.290346+0000","last_peered":"2024-09-05T13:15:52.290346+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:52.290346+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9356,"log_dups_size":0,"ondisk_log_size":9356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38313984,"num_objects":9354,"num_object_clones":0,"num_object_copies":18708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4935},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9426","reported_seq":9609,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.209524+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:52.209524+0000","last_peered":"2024-09-05T13:15:52.209524+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:52.209524+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9426,"log_dups_size":0,"ondisk_log_size":9426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38576128,"num_objects":9418,"num_object_clones":0,"num_object_copies":18836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5325,"num_write_kb":21300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4974},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9460","reported_seq":9594,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.248992+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:52.248992+0000","last_peered":"2024-09-05T13:15:52.248992+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:52.248992+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9460,"log_dups_size":0,"ondisk_log_size":9460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5590,"num_write_kb":22360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4890},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9519","reported_seq":9654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.266889+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:52.266889+0000","last_peered":"2024-09-05T13:15:52.266889+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:52.266889+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9519,"log_dups_size":0,"ondisk_log_size":9519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38941155,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6187,"num_write_kb":24748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4943},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":615375331,"num_objects":150239,"num_object_clones":0,"num_object_copies":300478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":150239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106038,"num_write_kb":424152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3930,"num_bytes_recovered":16089088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1251213312,"data_stored":1251130213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150320,"ondisk_log_size":150320,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687680,"kb_used_data":443568,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93684160,"statfs":{"total":96636764160,"available":95932579840,"internally_reserved":0,"allocated":454213632,"data_stored":453968542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":832300,"kb_used_data":521628,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93539540,"statfs":{"total":96636764160,"available":95784488960,"internally_reserved":0,"allocated":534147072,"data_stored":533897905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,1,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468740,"kb_used_data":256564,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93903100,"statfs":{"total":96636764160,"available":96156774400,"internally_reserved":0,"allocated":262721536,"data_stored":262400109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":534339584,"data_stored":534335971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262774784,"data_stored":262698911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454098944,"data_stored":454095331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:56.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 203 20 151880 151860 2.9214 1.76172 0.0188446 0.0267333 2024-09-05T13:15:57.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 204 20 152357 152337 2.91621 1.86328 0.0318742 0.0267829 2024-09-05T13:15:58.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 205 19 152829 152810 2.911 1.84766 0.0542091 0.02683 2024-09-05T13:15:58.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:59.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 206 20 153273 153253 2.90527 1.73047 0.0445924 0.0268838 2024-09-05T13:15:59.285 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:59.286 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:59.314 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-05T13:15:57.587313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620958198,"num_objects":151604,"num_object_clones":0,"num_object_copies":303208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":151604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107403,"num_write_kb":429606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3931,"num_bytes_recovered":16089107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151686,"ondisk_log_size":151686,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2000948,"kb_used_data":1228420,"kb_used_omap":12,"kb_used_meta":772467,"kb_avail":281114572,"statfs":{"total":289910292480,"available":287861321728,"internally_reserved":0,"allocated":1257902080,"data_stored":1257086396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":791006214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,7,9,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23982080,"num_objects":5855,"num_object_clones":0,"num_object_copies":11710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":5855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5855,"num_write_kb":23420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004234"},"pg_stats":[{"pgid":"2.f","version":"43'9501","reported_seq":9571,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.297018+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:57.297018+0000","last_peered":"2024-09-05T13:15:57.297018+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:57.297018+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916096,"num_objects":9501,"num_object_clones":0,"num_object_copies":19002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9501,"num_write_kb":38004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5080},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9314","reported_seq":9414,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.619152+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:52.619152+0000","last_peered":"2024-09-05T13:15:52.619152+0000","last_clean":"2024-09-05T13:15:52.619152+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:52.619152+0000","last_undegraded":"2024-09-05T13:15:52.619152+0000","last_fullsized":"2024-09-05T13:15:52.619152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9314,"log_dups_size":0,"ondisk_log_size":9314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38125568,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7324,"num_write_kb":29296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9502","reported_seq":9625,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.726635+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:52.726635+0000","last_peered":"2024-09-05T13:15:52.726635+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:52.726635+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38903808,"num_objects":9498,"num_object_clones":0,"num_object_copies":18996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7313,"num_write_kb":29252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5006}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9433","reported_seq":9498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:52.714979+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:52.714979+0000","last_peered":"2024-09-05T13:15:52.714979+0000","last_clean":"2024-09-05T13:15:52.714979+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:52.714979+0000","last_undegraded":"2024-09-05T13:15:52.714979+0000","last_fullsized":"2024-09-05T13:15:52.714979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38637568,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9433,"num_write_kb":37732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9501","reported_seq":9631,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.322231+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:57.322231+0000","last_peered":"2024-09-05T13:15:57.322231+0000","last_clean":"2024-09-05T13:15:57.322231+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:57.322231+0000","last_undegraded":"2024-09-05T13:15:57.322231+0000","last_fullsized":"2024-09-05T13:15:57.322231+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38887424,"num_objects":9494,"num_object_clones":0,"num_object_copies":18988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":24164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9620","reported_seq":9754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.272677+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:57.272677+0000","last_peered":"2024-09-05T13:15:57.272677+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:57.272677+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6075,"num_write_kb":24300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5164},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9384","reported_seq":9518,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.273635+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:57.273635+0000","last_peered":"2024-09-05T13:15:57.273635+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:57.273635+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5703,"num_write_kb":22812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9611","reported_seq":9738,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.243116+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:57.243116+0000","last_peered":"2024-09-05T13:15:57.243116+0000","last_clean":"2024-09-05T13:15:57.243116+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:57.243116+0000","last_undegraded":"2024-09-05T13:15:57.243116+0000","last_fullsized":"2024-09-05T13:15:57.243116+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9611,"log_dups_size":0,"ondisk_log_size":9611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39358464,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6221,"num_write_kb":24884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9404","reported_seq":9534,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.648049+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:52.648049+0000","last_peered":"2024-09-05T13:15:52.648049+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:52.648049+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9404,"log_dups_size":0,"ondisk_log_size":9404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38490112,"num_objects":9397,"num_object_clones":0,"num_object_copies":18794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":4946}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9532","reported_seq":9665,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.166954+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:57.166954+0000","last_peered":"2024-09-05T13:15:57.166954+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:57.166954+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39010304,"num_objects":9524,"num_object_clones":0,"num_object_copies":19048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5899,"num_write_kb":23596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5070},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9389","reported_seq":17355,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.735727+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:52.735727+0000","last_peered":"2024-09-05T13:15:52.735727+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:52.735727+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9389,"log_dups_size":0,"ondisk_log_size":9389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38436864,"num_objects":9384,"num_object_clones":0,"num_object_copies":18768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":9384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6984,"num_write_kb":27936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":460},{"shards":"2","objects":8922}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9212","reported_seq":9330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:52.672883+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:52.672883+0000","last_peered":"2024-09-05T13:15:52.672883+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:52.672883+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9212,"log_dups_size":0,"ondisk_log_size":9212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37707776,"num_objects":9206,"num_object_clones":0,"num_object_copies":18412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6721,"num_write_kb":26884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4819}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9485","reported_seq":9621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.303866+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:57.303866+0000","last_peered":"2024-09-05T13:15:57.303866+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:57.303866+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9485,"log_dups_size":0,"ondisk_log_size":9485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5636,"num_write_kb":22544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5064},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9570","reported_seq":9753,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.346923+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:57.346923+0000","last_peered":"2024-09-05T13:15:57.346923+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:57.346923+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9570,"log_dups_size":0,"ondisk_log_size":9570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39165952,"num_objects":9562,"num_object_clones":0,"num_object_copies":19124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5469,"num_write_kb":21876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5118},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9578","reported_seq":9711,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.351320+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:57.351320+0000","last_peered":"2024-09-05T13:15:57.351320+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:57.351320+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9578,"log_dups_size":0,"ondisk_log_size":9578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5708,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9647","reported_seq":9782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.276706+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:57.276706+0000","last_peered":"2024-09-05T13:15:57.276706+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:57.276706+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9647,"log_dups_size":0,"ondisk_log_size":9647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39465443,"num_objects":9636,"num_object_clones":0,"num_object_copies":19272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5071},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620958179,"num_objects":151602,"num_object_clones":0,"num_object_copies":303204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":460,"num_objects_unfound":0,"num_objects_dirty":151602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107401,"num_write_kb":429604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3930,"num_bytes_recovered":16089088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1257926656,"data_stored":1257843557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151683,"ondisk_log_size":151683,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687680,"kb_used_data":443568,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93684160,"statfs":{"total":96636764160,"available":95932579840,"internally_reserved":0,"allocated":454213632,"data_stored":453968542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":249949869},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844528,"kb_used_data":528288,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93527312,"statfs":{"total":96636764160,"available":95771967488,"internally_reserved":0,"allocated":540966912,"data_stored":540717745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,5,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468740,"kb_used_data":256564,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93903100,"statfs":{"total":96636764160,"available":96156774400,"internally_reserved":0,"allocated":262721536,"data_stored":262400109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541052928,"data_stored":541049315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262774784,"data_stored":262698911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454098944,"data_stored":454095331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:59.315 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:15:59.872 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:15:59.873 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:15:59.893 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-05T13:15:59.588164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":624570870,"num_objects":152486,"num_object_clones":0,"num_object_copies":304972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":308,"num_objects_unfound":0,"num_objects_dirty":152486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108285,"num_write_kb":433134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4083,"num_bytes_recovered":16711699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152568,"ondisk_log_size":152568,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009508,"kb_used_data":1232820,"kb_used_omap":12,"kb_used_meta":776627,"kb_avail":281106012,"statfs":{"total":289910292480,"available":287852556288,"internally_reserved":0,"allocated":1262407680,"data_stored":1261591996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":795266054},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,2,7,9,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-470,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5319,"num_write_kb":21276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":470,"num_bytes_recovered":1925120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004483"},"pg_stats":[{"pgid":"2.f","version":"43'9501","reported_seq":9571,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.297018+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:57.297018+0000","last_peered":"2024-09-05T13:15:57.297018+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:57.297018+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916096,"num_objects":9501,"num_object_clones":0,"num_object_copies":19002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9501,"num_write_kb":38004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5080},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9462","reported_seq":9561,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.770869+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:57.770869+0000","last_peered":"2024-09-05T13:15:57.770869+0000","last_clean":"2024-09-05T13:15:57.770869+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:57.770869+0000","last_undegraded":"2024-09-05T13:15:57.770869+0000","last_fullsized":"2024-09-05T13:15:57.770869+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9462,"log_dups_size":0,"ondisk_log_size":9462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7472,"num_write_kb":29888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9643","reported_seq":9766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.713746+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:57.713746+0000","last_peered":"2024-09-05T13:15:57.713746+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:57.713746+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9643,"log_dups_size":0,"ondisk_log_size":9643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39481344,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7454,"num_write_kb":29816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5147}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9569","reported_seq":9634,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.671989+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:57.671989+0000","last_peered":"2024-09-05T13:15:57.671989+0000","last_clean":"2024-09-05T13:15:57.671989+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:57.671989+0000","last_undegraded":"2024-09-05T13:15:57.671989+0000","last_fullsized":"2024-09-05T13:15:57.671989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9569,"log_dups_size":0,"ondisk_log_size":9569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9569,"num_write_kb":38276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9501","reported_seq":9631,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.322231+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:57.322231+0000","last_peered":"2024-09-05T13:15:57.322231+0000","last_clean":"2024-09-05T13:15:57.322231+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:57.322231+0000","last_undegraded":"2024-09-05T13:15:57.322231+0000","last_fullsized":"2024-09-05T13:15:57.322231+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38887424,"num_objects":9494,"num_object_clones":0,"num_object_copies":18988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":24164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9620","reported_seq":9754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.272677+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:57.272677+0000","last_peered":"2024-09-05T13:15:57.272677+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:57.272677+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6075,"num_write_kb":24300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5164},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9384","reported_seq":9518,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.273635+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:57.273635+0000","last_peered":"2024-09-05T13:15:57.273635+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:57.273635+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5703,"num_write_kb":22812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9611","reported_seq":9738,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.243116+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:57.243116+0000","last_peered":"2024-09-05T13:15:57.243116+0000","last_clean":"2024-09-05T13:15:57.243116+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:57.243116+0000","last_undegraded":"2024-09-05T13:15:57.243116+0000","last_fullsized":"2024-09-05T13:15:57.243116+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9611,"log_dups_size":0,"ondisk_log_size":9611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39358464,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6221,"num_write_kb":24884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9549","reported_seq":9679,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.745578+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:57.745578+0000","last_peered":"2024-09-05T13:15:57.745578+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:57.745578+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9549,"log_dups_size":0,"ondisk_log_size":9549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39084032,"num_objects":9542,"num_object_clones":0,"num_object_copies":19084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":28812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5091}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9532","reported_seq":9665,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.166954+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:57.166954+0000","last_peered":"2024-09-05T13:15:57.166954+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:57.166954+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39010304,"num_objects":9524,"num_object_clones":0,"num_object_copies":19048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5899,"num_write_kb":23596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5070},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9556","reported_seq":17826,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.768140+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:57.768140+0000","last_peered":"2024-09-05T13:15:57.768140+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:57.768140+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9556,"log_dups_size":0,"ondisk_log_size":9556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39120896,"num_objects":9551,"num_object_clones":0,"num_object_copies":19102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":308,"num_objects_unfound":0,"num_objects_dirty":9551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7151,"num_write_kb":28604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4079,"num_bytes_recovered":16699392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":308},{"shards":"2","objects":9241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9357","reported_seq":9475,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.704199+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:57.704199+0000","last_peered":"2024-09-05T13:15:57.704199+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:57.704199+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9357,"log_dups_size":0,"ondisk_log_size":9357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38301696,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6866,"num_write_kb":27464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4964}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9485","reported_seq":9621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.303866+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:57.303866+0000","last_peered":"2024-09-05T13:15:57.303866+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:57.303866+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9485,"log_dups_size":0,"ondisk_log_size":9485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5636,"num_write_kb":22544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5064},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9570","reported_seq":9753,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.346923+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:57.346923+0000","last_peered":"2024-09-05T13:15:57.346923+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:57.346923+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9570,"log_dups_size":0,"ondisk_log_size":9570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39165952,"num_objects":9562,"num_object_clones":0,"num_object_copies":19124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5469,"num_write_kb":21876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5118},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9578","reported_seq":9711,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.351320+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:57.351320+0000","last_peered":"2024-09-05T13:15:57.351320+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:57.351320+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9578,"log_dups_size":0,"ondisk_log_size":9578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5708,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9647","reported_seq":9782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.276706+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:57.276706+0000","last_peered":"2024-09-05T13:15:57.276706+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:57.276706+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9647,"log_dups_size":0,"ondisk_log_size":9647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39465443,"num_objects":9636,"num_object_clones":0,"num_object_copies":19272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5071},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":624570851,"num_objects":152484,"num_object_clones":0,"num_object_copies":304968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":308,"num_objects_unfound":0,"num_objects_dirty":152484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108283,"num_write_kb":433132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4082,"num_bytes_recovered":16711680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1262571520,"data_stored":1262488421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152565,"ondisk_log_size":152565,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696240,"kb_used_data":447968,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93675600,"statfs":{"total":96636764160,"available":95923814400,"internally_reserved":0,"allocated":458719232,"data_stored":458474142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844528,"kb_used_data":528288,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93527312,"statfs":{"total":96636764160,"available":95771967488,"internally_reserved":0,"allocated":540966912,"data_stored":540717745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,5,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468740,"kb_used_data":256564,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93903100,"statfs":{"total":96636764160,"available":96156774400,"internally_reserved":0,"allocated":262721536,"data_stored":262400109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541052928,"data_stored":541049315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262774784,"data_stored":262698911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":458743808,"data_stored":458740195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:15:59.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:15:59.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'9501", 'reported_seq': 9571, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.297018+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:15:57.297018+0000', 'last_peered': '2024-09-05T13:15:57.297018+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:15:57.297018+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 9501, 'log_dups_size': 0, 'ondisk_log_size': 9501, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38916096, 'num_objects': 9501, 'num_object_clones': 0, 'num_object_copies': 19002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9501, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9501, 'num_write_kb': 38004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5080}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:59.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:15:59.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'9643", 'reported_seq': 9766, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.713746+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:15:57.713746+0000', 'last_peered': '2024-09-05T13:15:57.713746+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:15:57.713746+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 9643, 'log_dups_size': 0, 'ondisk_log_size': 9643, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39481344, 'num_objects': 9639, 'num_object_clones': 0, 'num_object_copies': 19278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7454, 'num_write_kb': 29816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 5147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:59.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:15:59.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'9620", 'reported_seq': 9754, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.272677+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:15:57.272677+0000', 'last_peered': '2024-09-05T13:15:57.272677+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:15:57.272677+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 9620, 'log_dups_size': 0, 'ondisk_log_size': 9620, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39383040, 'num_objects': 9615, 'num_object_clones': 0, 'num_object_copies': 19230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6075, 'num_write_kb': 24300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5164}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:59.895 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'9384", 'reported_seq': 9518, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.273635+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:15:57.273635+0000', 'last_peered': '2024-09-05T13:15:57.273635+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:15:57.273635+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 9384, 'log_dups_size': 0, 'ondisk_log_size': 9384, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38412288, 'num_objects': 9378, 'num_object_clones': 0, 'num_object_copies': 18756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9378, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5703, 'num_write_kb': 22812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4942}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'9549", 'reported_seq': 9679, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.745578+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:15:57.745578+0000', 'last_peered': '2024-09-05T13:15:57.745578+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:15:57.745578+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 9549, 'log_dups_size': 0, 'ondisk_log_size': 9549, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39084032, 'num_objects': 9542, 'num_object_clones': 0, 'num_object_copies': 19084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9542, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7203, 'num_write_kb': 28812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 5091}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'9532", 'reported_seq': 9665, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.166954+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:15:57.166954+0000', 'last_peered': '2024-09-05T13:15:57.166954+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:15:57.166954+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 9532, 'log_dups_size': 0, 'ondisk_log_size': 9532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39010304, 'num_objects': 9524, 'num_object_clones': 0, 'num_object_copies': 19048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5899, 'num_write_kb': 23596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5070}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'9556", 'reported_seq': 17826, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.768140+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:15:57.768140+0000', 'last_peered': '2024-09-05T13:15:57.768140+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:15:57.768140+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 9556, 'log_dups_size': 0, 'ondisk_log_size': 9556, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39120896, 'num_objects': 9551, 'num_object_clones': 0, 'num_object_copies': 19102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 308, 'num_objects_unfound': 0, 'num_objects_dirty': 9551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7151, 'num_write_kb': 28604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4079, 'num_bytes_recovered': 16699392, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 308}, {'shards': '2', 'objects': 9241}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'9357", 'reported_seq': 9475, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.704199+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:15:57.704199+0000', 'last_peered': '2024-09-05T13:15:57.704199+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:15:57.704199+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 9357, 'log_dups_size': 0, 'ondisk_log_size': 9357, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38301696, 'num_objects': 9351, 'num_object_clones': 0, 'num_object_copies': 18702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9351, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6866, 'num_write_kb': 27464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 4964}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:15:59.896 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:15:59.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'9485", 'reported_seq': 9621, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.303866+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:15:57.303866+0000', 'last_peered': '2024-09-05T13:15:57.303866+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:15:57.303866+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 9485, 'log_dups_size': 0, 'ondisk_log_size': 9485, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38842368, 'num_objects': 9483, 'num_object_clones': 0, 'num_object_copies': 18966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5636, 'num_write_kb': 22544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5064}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:59.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:15:59.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'9570", 'reported_seq': 9753, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.346923+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:15:57.346923+0000', 'last_peered': '2024-09-05T13:15:57.346923+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:15:57.346923+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 9570, 'log_dups_size': 0, 'ondisk_log_size': 9570, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39165952, 'num_objects': 9562, 'num_object_clones': 0, 'num_object_copies': 19124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9562, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5469, 'num_write_kb': 21876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5118}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:59.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:15:59.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'9578", 'reported_seq': 9711, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.351320+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:15:57.351320+0000', 'last_peered': '2024-09-05T13:15:57.351320+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:15:57.351320+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 9578, 'log_dups_size': 0, 'ondisk_log_size': 9578, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39215104, 'num_objects': 9574, 'num_object_clones': 0, 'num_object_copies': 19148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5708, 'num_write_kb': 22832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5008}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:59.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:15:59.897 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'9647", 'reported_seq': 9782, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:15:57.276706+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:15:57.276706+0000', 'last_peered': '2024-09-05T13:15:57.276706+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:15:57.276706+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 9647, 'log_dups_size': 0, 'ondisk_log_size': 9647, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39465443, 'num_objects': 9636, 'num_object_clones': 0, 'num_object_copies': 19272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6315, 'num_write_kb': 25260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5071}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:15:59.897 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:00.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 207 20 153684 153664 2.89899 1.60547 0.0703875 0.0269416 2024-09-05T13:16:00.594 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:00.594 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/304972 objects degraded (16.102%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":287,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152486,"data_bytes":624570870,"bytes_used":2057736192,"bytes_avail":287852556288,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":304972,"degraded_ratio":0.16102461865351572,"misplaced_objects":308,"misplaced_total":304972,"misplaced_ratio":0.0010099287803470483,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":160366,"recovering_keys_per_sec":0,"num_objects_recovered":470,"num_bytes_recovered":1925120,"num_keys_recovered":0,"write_bytes_sec":1814873,"read_op_per_sec":0,"write_op_per_sec":443},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:00.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:00.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:01.185 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:01.212 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-05T13:15:59.588164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":624570870,"num_objects":152486,"num_object_clones":0,"num_object_copies":304972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":308,"num_objects_unfound":0,"num_objects_dirty":152486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108285,"num_write_kb":433134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4083,"num_bytes_recovered":16711699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152568,"ondisk_log_size":152568,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009508,"kb_used_data":1232820,"kb_used_omap":12,"kb_used_meta":776627,"kb_avail":281106012,"statfs":{"total":289910292480,"available":287852556288,"internally_reserved":0,"allocated":1262407680,"data_stored":1261591996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":795266054},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,2,7,9,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-470,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5319,"num_write_kb":21276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":470,"num_bytes_recovered":1925120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004483"},"pg_stats":[{"pgid":"2.f","version":"43'9501","reported_seq":9571,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.297018+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:15:57.297018+0000","last_peered":"2024-09-05T13:15:57.297018+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:15:57.297018+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916096,"num_objects":9501,"num_object_clones":0,"num_object_copies":19002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9501,"num_write_kb":38004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5080},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9462","reported_seq":9561,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.770869+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:15:57.770869+0000","last_peered":"2024-09-05T13:15:57.770869+0000","last_clean":"2024-09-05T13:15:57.770869+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:15:57.770869+0000","last_undegraded":"2024-09-05T13:15:57.770869+0000","last_fullsized":"2024-09-05T13:15:57.770869+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9462,"log_dups_size":0,"ondisk_log_size":9462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7472,"num_write_kb":29888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9643","reported_seq":9766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.713746+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:15:57.713746+0000","last_peered":"2024-09-05T13:15:57.713746+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:15:57.713746+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9643,"log_dups_size":0,"ondisk_log_size":9643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39481344,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7454,"num_write_kb":29816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5147}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9569","reported_seq":9634,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.671989+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:15:57.671989+0000","last_peered":"2024-09-05T13:15:57.671989+0000","last_clean":"2024-09-05T13:15:57.671989+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:15:57.671989+0000","last_undegraded":"2024-09-05T13:15:57.671989+0000","last_fullsized":"2024-09-05T13:15:57.671989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9569,"log_dups_size":0,"ondisk_log_size":9569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9569,"num_write_kb":38276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9501","reported_seq":9631,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.322231+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:15:57.322231+0000","last_peered":"2024-09-05T13:15:57.322231+0000","last_clean":"2024-09-05T13:15:57.322231+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:15:57.322231+0000","last_undegraded":"2024-09-05T13:15:57.322231+0000","last_fullsized":"2024-09-05T13:15:57.322231+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38887424,"num_objects":9494,"num_object_clones":0,"num_object_copies":18988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":24164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9620","reported_seq":9754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.272677+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:15:57.272677+0000","last_peered":"2024-09-05T13:15:57.272677+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:15:57.272677+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6075,"num_write_kb":24300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5164},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9384","reported_seq":9518,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.273635+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:15:57.273635+0000","last_peered":"2024-09-05T13:15:57.273635+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:15:57.273635+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9384,"log_dups_size":0,"ondisk_log_size":9384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5703,"num_write_kb":22812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9611","reported_seq":9738,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:15:57.243116+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:15:57.243116+0000","last_peered":"2024-09-05T13:15:57.243116+0000","last_clean":"2024-09-05T13:15:57.243116+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:15:57.243116+0000","last_undegraded":"2024-09-05T13:15:57.243116+0000","last_fullsized":"2024-09-05T13:15:57.243116+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9611,"log_dups_size":0,"ondisk_log_size":9611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39358464,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6221,"num_write_kb":24884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9549","reported_seq":9679,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.745578+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:15:57.745578+0000","last_peered":"2024-09-05T13:15:57.745578+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:15:57.745578+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9549,"log_dups_size":0,"ondisk_log_size":9549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39084032,"num_objects":9542,"num_object_clones":0,"num_object_copies":19084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7203,"num_write_kb":28812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5091}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9532","reported_seq":9665,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.166954+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:15:57.166954+0000","last_peered":"2024-09-05T13:15:57.166954+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:15:57.166954+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9532,"log_dups_size":0,"ondisk_log_size":9532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39010304,"num_objects":9524,"num_object_clones":0,"num_object_copies":19048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5899,"num_write_kb":23596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5070},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9556","reported_seq":17826,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.768140+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:15:57.768140+0000","last_peered":"2024-09-05T13:15:57.768140+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:15:57.768140+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9556,"log_dups_size":0,"ondisk_log_size":9556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39120896,"num_objects":9551,"num_object_clones":0,"num_object_copies":19102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":308,"num_objects_unfound":0,"num_objects_dirty":9551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7151,"num_write_kb":28604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4079,"num_bytes_recovered":16699392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":308},{"shards":"2","objects":9241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9357","reported_seq":9475,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.704199+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:15:57.704199+0000","last_peered":"2024-09-05T13:15:57.704199+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:15:57.704199+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9357,"log_dups_size":0,"ondisk_log_size":9357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38301696,"num_objects":9351,"num_object_clones":0,"num_object_copies":18702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6866,"num_write_kb":27464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":4964}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9485","reported_seq":9621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.303866+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:15:57.303866+0000","last_peered":"2024-09-05T13:15:57.303866+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:15:57.303866+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9485,"log_dups_size":0,"ondisk_log_size":9485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5636,"num_write_kb":22544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5064},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9570","reported_seq":9753,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.346923+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:15:57.346923+0000","last_peered":"2024-09-05T13:15:57.346923+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:15:57.346923+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9570,"log_dups_size":0,"ondisk_log_size":9570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39165952,"num_objects":9562,"num_object_clones":0,"num_object_copies":19124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5469,"num_write_kb":21876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5118},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9578","reported_seq":9711,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.351320+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:15:57.351320+0000","last_peered":"2024-09-05T13:15:57.351320+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:15:57.351320+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9578,"log_dups_size":0,"ondisk_log_size":9578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5708,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5008},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9647","reported_seq":9782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:15:57.276706+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:15:57.276706+0000","last_peered":"2024-09-05T13:15:57.276706+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:15:57.276706+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9647,"log_dups_size":0,"ondisk_log_size":9647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39465443,"num_objects":9636,"num_object_clones":0,"num_object_copies":19272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5071},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":624570851,"num_objects":152484,"num_object_clones":0,"num_object_copies":304968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":308,"num_objects_unfound":0,"num_objects_dirty":152484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108283,"num_write_kb":433132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4082,"num_bytes_recovered":16711680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1262571520,"data_stored":1262488421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152565,"ondisk_log_size":152565,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696240,"kb_used_data":447968,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93675600,"statfs":{"total":96636764160,"available":95923814400,"internally_reserved":0,"allocated":458719232,"data_stored":458474142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":254209709},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844528,"kb_used_data":528288,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93527312,"statfs":{"total":96636764160,"available":95771967488,"internally_reserved":0,"allocated":540966912,"data_stored":540717745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,5,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468740,"kb_used_data":256564,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93903100,"statfs":{"total":96636764160,"available":96156774400,"internally_reserved":0,"allocated":262721536,"data_stored":262400109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541052928,"data_stored":541049315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":262774784,"data_stored":262698911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":458743808,"data_stored":458740195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:01.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 208 20 154093 154073 2.89274 1.59766 0.0796326 0.0269996 2024-09-05T13:16:02.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 209 20 154565 154545 2.88772 1.84375 0.0170975 0.0270475 2024-09-05T13:16:03.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 210 20 155026 155006 2.88254 1.80078 0.0262423 0.0270948 2024-09-05T13:16:04.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:04.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 211 20 155485 155465 2.87738 1.79297 0.0166646 0.0271448 2024-09-05T13:16:04.803 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:04.803 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:04.832 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-05T13:16:03.589743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633614838,"num_objects":154694,"num_object_clones":0,"num_object_copies":309388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":154694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110493,"num_write_kb":441966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4225,"num_bytes_recovered":17293331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154776,"ondisk_log_size":154776,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2040452,"kb_used_data":1251284,"kb_used_omap":12,"kb_used_meta":789107,"kb_avail":281075068,"statfs":{"total":289910292480,"available":287820869632,"internally_reserved":0,"allocated":1281314816,"data_stored":1280499132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":808045574},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,4,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27275264,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-463,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6659,"num_write_kb":26636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004504"},"pg_stats":[{"pgid":"2.f","version":"43'9639","reported_seq":9709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.379922+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:02.379922+0000","last_peered":"2024-09-05T13:16:02.379922+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:02.379922+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39481344,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9639,"num_write_kb":38556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9596","reported_seq":9696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.772508+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:02.772508+0000","last_peered":"2024-09-05T13:16:02.772508+0000","last_clean":"2024-09-05T13:16:02.772508+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:02.772508+0000","last_undegraded":"2024-09-05T13:16:02.772508+0000","last_fullsized":"2024-09-05T13:16:02.772508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9596,"log_dups_size":0,"ondisk_log_size":9596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39280640,"num_objects":9590,"num_object_clones":0,"num_object_copies":19180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7606,"num_write_kb":30424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9787","reported_seq":9910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.743133+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:02.743133+0000","last_peered":"2024-09-05T13:16:02.743133+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:02.743133+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9787,"log_dups_size":0,"ondisk_log_size":9787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40071168,"num_objects":9783,"num_object_clones":0,"num_object_copies":19566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7598,"num_write_kb":30392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9701","reported_seq":9766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.689010+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:02.689010+0000","last_peered":"2024-09-05T13:16:02.689010+0000","last_clean":"2024-09-05T13:16:02.689010+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:02.689010+0000","last_undegraded":"2024-09-05T13:16:02.689010+0000","last_fullsized":"2024-09-05T13:16:02.689010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":38804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9624","reported_seq":9754,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.346998+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:02.346998+0000","last_peered":"2024-09-05T13:16:02.346998+0000","last_clean":"2024-09-05T13:16:02.346998+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:02.346998+0000","last_undegraded":"2024-09-05T13:16:02.346998+0000","last_fullsized":"2024-09-05T13:16:02.346998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9624,"log_dups_size":0,"ondisk_log_size":9624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39391232,"num_objects":9617,"num_object_clones":0,"num_object_copies":19234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6164,"num_write_kb":24656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9751","reported_seq":9884,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.368917+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:02.368917+0000","last_peered":"2024-09-05T13:16:02.368917+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:02.368917+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9751,"log_dups_size":0,"ondisk_log_size":9751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39919616,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6206,"num_write_kb":24824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9520","reported_seq":9655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.349812+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:16:02.349812+0000","last_peered":"2024-09-05T13:16:02.349812+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:02.349812+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969344,"num_objects":9514,"num_object_clones":0,"num_object_copies":19028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9767","reported_seq":9894,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.362743+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:02.362743+0000","last_peered":"2024-09-05T13:16:02.362743+0000","last_clean":"2024-09-05T13:16:02.362743+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:02.362743+0000","last_undegraded":"2024-09-05T13:16:02.362743+0000","last_fullsized":"2024-09-05T13:16:02.362743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9767,"log_dups_size":0,"ondisk_log_size":9767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39997440,"num_objects":9765,"num_object_clones":0,"num_object_copies":19530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6377,"num_write_kb":25508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9686","reported_seq":9816,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.759129+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:02.759129+0000","last_peered":"2024-09-05T13:16:02.759129+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:02.759129+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9686,"log_dups_size":0,"ondisk_log_size":9686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7340,"num_write_kb":29360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5228}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9665","reported_seq":9798,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.320869+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:02.320869+0000","last_peered":"2024-09-05T13:16:02.320869+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:02.320869+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9665,"log_dups_size":0,"ondisk_log_size":9665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39555072,"num_objects":9657,"num_object_clones":0,"num_object_copies":19314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5203},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9702","reported_seq":18253,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.741529+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:16:02.741529+0000","last_peered":"2024-09-05T13:16:02.741529+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:16:02.741529+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9702,"log_dups_size":0,"ondisk_log_size":9702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4221,"num_bytes_recovered":17281024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":166},{"shards":"2","objects":9529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9506","reported_seq":9624,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.731402+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:02.731402+0000","last_peered":"2024-09-05T13:16:02.731402+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:02.731402+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9506,"log_dups_size":0,"ondisk_log_size":9506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7015,"num_write_kb":28060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5113}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9610","reported_seq":9746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.321966+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:02.321966+0000","last_peered":"2024-09-05T13:16:02.321966+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:02.321966+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9610,"log_dups_size":0,"ondisk_log_size":9610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354368,"num_objects":9608,"num_object_clones":0,"num_object_copies":19216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5761,"num_write_kb":23044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5189},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9714","reported_seq":9897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.360155+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:02.360155+0000","last_peered":"2024-09-05T13:16:02.360155+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:02.360155+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9714,"log_dups_size":0,"ondisk_log_size":9714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39755776,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5613,"num_write_kb":22452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9723","reported_seq":9855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.320115+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:02.320115+0000","last_peered":"2024-09-05T13:16:02.320115+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:02.320115+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9723,"log_dups_size":0,"ondisk_log_size":9723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39809024,"num_objects":9719,"num_object_clones":0,"num_object_copies":19438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5853,"num_write_kb":23412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5153},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9782","reported_seq":9916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.362556+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:02.362556+0000","last_peered":"2024-09-05T13:16:02.362556+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:02.362556+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9782,"log_dups_size":0,"ondisk_log_size":9782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6450,"num_write_kb":25800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633614819,"num_objects":154692,"num_object_clones":0,"num_object_copies":309384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":154692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110491,"num_write_kb":441964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4224,"num_bytes_recovered":17293312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281482752,"data_stored":1281399653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154773,"ondisk_log_size":154773,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704972,"kb_used_data":452540,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93666868,"statfs":{"total":96636764160,"available":95914872832,"internally_reserved":0,"allocated":463400960,"data_stored":463155870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855276,"kb_used_data":534876,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93516564,"statfs":{"total":96636764160,"available":95760961536,"internally_reserved":0,"allocated":547713024,"data_stored":547463857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480204,"kb_used_data":263868,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93891636,"statfs":{"total":96636764160,"available":96145035264,"internally_reserved":0,"allocated":270200832,"data_stored":269879405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547807232,"data_stored":547803619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":270301087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463298560,"data_stored":463294947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:04.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:05.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 212 20 155886 155866 2.8712 1.56641 0.0278793 0.0272015 2024-09-05T13:16:05.438 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:05.439 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:05.466 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-05T13:16:03.589743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633614838,"num_objects":154694,"num_object_clones":0,"num_object_copies":309388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":154694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110493,"num_write_kb":441966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4225,"num_bytes_recovered":17293331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154776,"ondisk_log_size":154776,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2040452,"kb_used_data":1251284,"kb_used_omap":12,"kb_used_meta":789107,"kb_avail":281075068,"statfs":{"total":289910292480,"available":287820869632,"internally_reserved":0,"allocated":1281314816,"data_stored":1280499132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":808045574},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,4,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27275264,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-463,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6659,"num_write_kb":26636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":1896448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004504"},"pg_stats":[{"pgid":"2.f","version":"43'9639","reported_seq":9709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.379922+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:02.379922+0000","last_peered":"2024-09-05T13:16:02.379922+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:02.379922+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39481344,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9639,"num_write_kb":38556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9596","reported_seq":9696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.772508+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:02.772508+0000","last_peered":"2024-09-05T13:16:02.772508+0000","last_clean":"2024-09-05T13:16:02.772508+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:02.772508+0000","last_undegraded":"2024-09-05T13:16:02.772508+0000","last_fullsized":"2024-09-05T13:16:02.772508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9596,"log_dups_size":0,"ondisk_log_size":9596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39280640,"num_objects":9590,"num_object_clones":0,"num_object_copies":19180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7606,"num_write_kb":30424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9787","reported_seq":9910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.743133+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:02.743133+0000","last_peered":"2024-09-05T13:16:02.743133+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:02.743133+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9787,"log_dups_size":0,"ondisk_log_size":9787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40071168,"num_objects":9783,"num_object_clones":0,"num_object_copies":19566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7598,"num_write_kb":30392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9701","reported_seq":9766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.689010+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:02.689010+0000","last_peered":"2024-09-05T13:16:02.689010+0000","last_clean":"2024-09-05T13:16:02.689010+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:02.689010+0000","last_undegraded":"2024-09-05T13:16:02.689010+0000","last_fullsized":"2024-09-05T13:16:02.689010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":38804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9624","reported_seq":9754,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.346998+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:02.346998+0000","last_peered":"2024-09-05T13:16:02.346998+0000","last_clean":"2024-09-05T13:16:02.346998+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:02.346998+0000","last_undegraded":"2024-09-05T13:16:02.346998+0000","last_fullsized":"2024-09-05T13:16:02.346998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9624,"log_dups_size":0,"ondisk_log_size":9624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39391232,"num_objects":9617,"num_object_clones":0,"num_object_copies":19234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6164,"num_write_kb":24656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9751","reported_seq":9884,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.368917+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:02.368917+0000","last_peered":"2024-09-05T13:16:02.368917+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:02.368917+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9751,"log_dups_size":0,"ondisk_log_size":9751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39919616,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6206,"num_write_kb":24824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9520","reported_seq":9655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.349812+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:16:02.349812+0000","last_peered":"2024-09-05T13:16:02.349812+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:02.349812+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969344,"num_objects":9514,"num_object_clones":0,"num_object_copies":19028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9767","reported_seq":9894,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.362743+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:02.362743+0000","last_peered":"2024-09-05T13:16:02.362743+0000","last_clean":"2024-09-05T13:16:02.362743+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:02.362743+0000","last_undegraded":"2024-09-05T13:16:02.362743+0000","last_fullsized":"2024-09-05T13:16:02.362743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9767,"log_dups_size":0,"ondisk_log_size":9767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39997440,"num_objects":9765,"num_object_clones":0,"num_object_copies":19530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6377,"num_write_kb":25508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9686","reported_seq":9816,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.759129+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:02.759129+0000","last_peered":"2024-09-05T13:16:02.759129+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:02.759129+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9686,"log_dups_size":0,"ondisk_log_size":9686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7340,"num_write_kb":29360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5228}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9665","reported_seq":9798,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.320869+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:02.320869+0000","last_peered":"2024-09-05T13:16:02.320869+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:02.320869+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9665,"log_dups_size":0,"ondisk_log_size":9665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39555072,"num_objects":9657,"num_object_clones":0,"num_object_copies":19314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5203},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9702","reported_seq":18253,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.741529+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:16:02.741529+0000","last_peered":"2024-09-05T13:16:02.741529+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:16:02.741529+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9702,"log_dups_size":0,"ondisk_log_size":9702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4221,"num_bytes_recovered":17281024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":166},{"shards":"2","objects":9529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9506","reported_seq":9624,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.731402+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:02.731402+0000","last_peered":"2024-09-05T13:16:02.731402+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:02.731402+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9506,"log_dups_size":0,"ondisk_log_size":9506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7015,"num_write_kb":28060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5113}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9610","reported_seq":9746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.321966+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:02.321966+0000","last_peered":"2024-09-05T13:16:02.321966+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:02.321966+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9610,"log_dups_size":0,"ondisk_log_size":9610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354368,"num_objects":9608,"num_object_clones":0,"num_object_copies":19216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5761,"num_write_kb":23044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5189},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9714","reported_seq":9897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.360155+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:02.360155+0000","last_peered":"2024-09-05T13:16:02.360155+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:02.360155+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9714,"log_dups_size":0,"ondisk_log_size":9714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39755776,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5613,"num_write_kb":22452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9723","reported_seq":9855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.320115+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:02.320115+0000","last_peered":"2024-09-05T13:16:02.320115+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:02.320115+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9723,"log_dups_size":0,"ondisk_log_size":9723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39809024,"num_objects":9719,"num_object_clones":0,"num_object_copies":19438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5853,"num_write_kb":23412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5153},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9782","reported_seq":9916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.362556+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:02.362556+0000","last_peered":"2024-09-05T13:16:02.362556+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:02.362556+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9782,"log_dups_size":0,"ondisk_log_size":9782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6450,"num_write_kb":25800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633614819,"num_objects":154692,"num_object_clones":0,"num_object_copies":309384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":154692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110491,"num_write_kb":441964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4224,"num_bytes_recovered":17293312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281482752,"data_stored":1281399653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154773,"ondisk_log_size":154773,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704972,"kb_used_data":452540,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93666868,"statfs":{"total":96636764160,"available":95914872832,"internally_reserved":0,"allocated":463400960,"data_stored":463155870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855276,"kb_used_data":534876,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93516564,"statfs":{"total":96636764160,"available":95760961536,"internally_reserved":0,"allocated":547713024,"data_stored":547463857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480204,"kb_used_data":263868,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93891636,"statfs":{"total":96636764160,"available":96145035264,"internally_reserved":0,"allocated":270200832,"data_stored":269879405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547807232,"data_stored":547803619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":270376960,"data_stored":270301087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463298560,"data_stored":463294947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:05.467 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'9639", 'reported_seq': 9709, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.379922+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:02.379922+0000', 'last_peered': '2024-09-05T13:16:02.379922+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:02.379922+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 9639, 'log_dups_size': 0, 'ondisk_log_size': 9639, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39481344, 'num_objects': 9639, 'num_object_clones': 0, 'num_object_copies': 19278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9639, 'num_write_kb': 38556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5218}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'9787", 'reported_seq': 9910, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.743133+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:02.743133+0000', 'last_peered': '2024-09-05T13:16:02.743133+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:02.743133+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 9787, 'log_dups_size': 0, 'ondisk_log_size': 9787, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40071168, 'num_objects': 9783, 'num_object_clones': 0, 'num_object_copies': 19566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9783, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7598, 'num_write_kb': 30392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 5291}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'9751", 'reported_seq': 9884, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.368917+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:02.368917+0000', 'last_peered': '2024-09-05T13:16:02.368917+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:02.368917+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 9751, 'log_dups_size': 0, 'ondisk_log_size': 9751, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39919616, 'num_objects': 9746, 'num_object_clones': 0, 'num_object_copies': 19492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9746, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6206, 'num_write_kb': 24824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5295}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'9520", 'reported_seq': 9655, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.349812+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:16:02.349812+0000', 'last_peered': '2024-09-05T13:16:02.349812+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:02.349812+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 9520, 'log_dups_size': 0, 'ondisk_log_size': 9520, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38969344, 'num_objects': 9514, 'num_object_clones': 0, 'num_object_copies': 19028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5839, 'num_write_kb': 23356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5078}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:05.468 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'9686", 'reported_seq': 9816, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.759129+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:02.759129+0000', 'last_peered': '2024-09-05T13:16:02.759129+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:02.759129+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 9686, 'log_dups_size': 0, 'ondisk_log_size': 9686, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39645184, 'num_objects': 9679, 'num_object_clones': 0, 'num_object_copies': 19358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7340, 'num_write_kb': 29360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 5228}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'9665", 'reported_seq': 9798, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.320869+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:02.320869+0000', 'last_peered': '2024-09-05T13:16:02.320869+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:02.320869+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 9665, 'log_dups_size': 0, 'ondisk_log_size': 9665, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39555072, 'num_objects': 9657, 'num_object_clones': 0, 'num_object_copies': 19314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6032, 'num_write_kb': 24128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5203}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'9702", 'reported_seq': 18253, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.741529+0000', 'last_change': '2024-09-05T13:13:18.470629+0000', 'last_active': '2024-09-05T13:16:02.741529+0000', 'last_peered': '2024-09-05T13:16:02.741529+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:16:02.741529+0000', 'last_undegraded': '2024-09-05T13:13:17.954749+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 9702, 'log_dups_size': 0, 'ondisk_log_size': 9702, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39718912, 'num_objects': 9697, 'num_object_clones': 0, 'num_object_copies': 19394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 166, 'num_objects_unfound': 0, 'num_objects_dirty': 9697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7297, 'num_write_kb': 29188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4221, 'num_bytes_recovered': 17281024, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 166}, {'shards': '2', 'objects': 9529}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'9506", 'reported_seq': 9624, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.731402+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:02.731402+0000', 'last_peered': '2024-09-05T13:16:02.731402+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:02.731402+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 9506, 'log_dups_size': 0, 'ondisk_log_size': 9506, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38912000, 'num_objects': 9500, 'num_object_clones': 0, 'num_object_copies': 19000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7015, 'num_write_kb': 28060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 5113}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'9610", 'reported_seq': 9746, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.321966+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:02.321966+0000', 'last_peered': '2024-09-05T13:16:02.321966+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:02.321966+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 9610, 'log_dups_size': 0, 'ondisk_log_size': 9610, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39354368, 'num_objects': 9608, 'num_object_clones': 0, 'num_object_copies': 19216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9608, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5761, 'num_write_kb': 23044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5189}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'9714", 'reported_seq': 9897, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.360155+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:02.360155+0000', 'last_peered': '2024-09-05T13:16:02.360155+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:02.360155+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 9714, 'log_dups_size': 0, 'ondisk_log_size': 9714, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39755776, 'num_objects': 9706, 'num_object_clones': 0, 'num_object_copies': 19412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5613, 'num_write_kb': 22452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5262}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'9723", 'reported_seq': 9855, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.320115+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:02.320115+0000', 'last_peered': '2024-09-05T13:16:02.320115+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:02.320115+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 9723, 'log_dups_size': 0, 'ondisk_log_size': 9723, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39809024, 'num_objects': 9719, 'num_object_clones': 0, 'num_object_copies': 19438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9719, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5853, 'num_write_kb': 23412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5153}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'9782", 'reported_seq': 9916, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:02.362556+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:02.362556+0000', 'last_peered': '2024-09-05T13:16:02.362556+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:02.362556+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 9782, 'log_dups_size': 0, 'ondisk_log_size': 9782, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40018403, 'num_objects': 9771, 'num_object_clones': 0, 'num_object_copies': 19542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6450, 'num_write_kb': 25800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5206}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:05.470 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:06.169 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:06.169 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49108/309388 objects degraded (15.873%), 12 pgs degraded, 12 pgs undersized","count":24},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":293,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":154694,"data_bytes":633614838,"bytes_used":2089422848,"bytes_avail":287820869632,"bytes_total":289910292480,"degraded_objects":49108,"degraded_total":309388,"degraded_ratio":0.1587262595834357,"misplaced_objects":166,"misplaced_total":309388,"misplaced_ratio":0.00053654311091574339,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":157978,"recovering_keys_per_sec":0,"num_objects_recovered":463,"num_bytes_recovered":1896448,"num_keys_recovered":0,"write_bytes_sec":2272085,"read_op_per_sec":0,"write_op_per_sec":554},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:06.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:06.188 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:06.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 213 20 156300 156280 2.86531 1.61719 0.0613452 0.0272582 2024-09-05T13:16:06.779 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:06.779 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:06.808 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-05T13:16:05.590733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633614838,"num_objects":154694,"num_object_clones":0,"num_object_copies":309388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":154694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110493,"num_write_kb":441966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4225,"num_bytes_recovered":17293331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154776,"ondisk_log_size":154776,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2070124,"kb_used_data":1258748,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281045396,"statfs":{"total":289910292480,"available":287790485504,"internally_reserved":0,"allocated":1288957952,"data_stored":1288142268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":830786566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,4,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18239488,"num_objects":4453,"num_object_clones":0,"num_object_copies":8906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-294,"num_objects_unfound":0,"num_objects_dirty":4453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4453,"num_write_kb":17812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":1204224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004850"},"pg_stats":[{"pgid":"2.f","version":"43'9639","reported_seq":9709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.379922+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:02.379922+0000","last_peered":"2024-09-05T13:16:02.379922+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:02.379922+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9639,"log_dups_size":0,"ondisk_log_size":9639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39481344,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9639,"num_write_kb":38556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9596","reported_seq":9696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.772508+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:02.772508+0000","last_peered":"2024-09-05T13:16:02.772508+0000","last_clean":"2024-09-05T13:16:02.772508+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:02.772508+0000","last_undegraded":"2024-09-05T13:16:02.772508+0000","last_fullsized":"2024-09-05T13:16:02.772508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9596,"log_dups_size":0,"ondisk_log_size":9596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39280640,"num_objects":9590,"num_object_clones":0,"num_object_copies":19180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7606,"num_write_kb":30424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9787","reported_seq":9910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.743133+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:02.743133+0000","last_peered":"2024-09-05T13:16:02.743133+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:02.743133+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9787,"log_dups_size":0,"ondisk_log_size":9787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40071168,"num_objects":9783,"num_object_clones":0,"num_object_copies":19566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7598,"num_write_kb":30392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9701","reported_seq":9766,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.689010+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:02.689010+0000","last_peered":"2024-09-05T13:16:02.689010+0000","last_clean":"2024-09-05T13:16:02.689010+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:02.689010+0000","last_undegraded":"2024-09-05T13:16:02.689010+0000","last_fullsized":"2024-09-05T13:16:02.689010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39735296,"num_objects":9701,"num_object_clones":0,"num_object_copies":19402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9701,"num_write_kb":38804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9624","reported_seq":9754,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.346998+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:02.346998+0000","last_peered":"2024-09-05T13:16:02.346998+0000","last_clean":"2024-09-05T13:16:02.346998+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:02.346998+0000","last_undegraded":"2024-09-05T13:16:02.346998+0000","last_fullsized":"2024-09-05T13:16:02.346998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9624,"log_dups_size":0,"ondisk_log_size":9624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39391232,"num_objects":9617,"num_object_clones":0,"num_object_copies":19234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6164,"num_write_kb":24656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9751","reported_seq":9884,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.368917+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:02.368917+0000","last_peered":"2024-09-05T13:16:02.368917+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:02.368917+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9751,"log_dups_size":0,"ondisk_log_size":9751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39919616,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6206,"num_write_kb":24824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9520","reported_seq":9655,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.349812+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:16:02.349812+0000","last_peered":"2024-09-05T13:16:02.349812+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:02.349812+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969344,"num_objects":9514,"num_object_clones":0,"num_object_copies":19028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9767","reported_seq":9894,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:02.362743+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:02.362743+0000","last_peered":"2024-09-05T13:16:02.362743+0000","last_clean":"2024-09-05T13:16:02.362743+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:02.362743+0000","last_undegraded":"2024-09-05T13:16:02.362743+0000","last_fullsized":"2024-09-05T13:16:02.362743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9767,"log_dups_size":0,"ondisk_log_size":9767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39997440,"num_objects":9765,"num_object_clones":0,"num_object_copies":19530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6377,"num_write_kb":25508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9686","reported_seq":9816,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.759129+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:02.759129+0000","last_peered":"2024-09-05T13:16:02.759129+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:02.759129+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9686,"log_dups_size":0,"ondisk_log_size":9686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39645184,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7340,"num_write_kb":29360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5228}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9665","reported_seq":9798,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.320869+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:02.320869+0000","last_peered":"2024-09-05T13:16:02.320869+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:02.320869+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9665,"log_dups_size":0,"ondisk_log_size":9665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39555072,"num_objects":9657,"num_object_clones":0,"num_object_copies":19314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5203},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9702","reported_seq":18253,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.741529+0000","last_change":"2024-09-05T13:13:18.470629+0000","last_active":"2024-09-05T13:16:02.741529+0000","last_peered":"2024-09-05T13:16:02.741529+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:16:02.741529+0000","last_undegraded":"2024-09-05T13:13:17.954749+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9702,"log_dups_size":0,"ondisk_log_size":9702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39718912,"num_objects":9697,"num_object_clones":0,"num_object_copies":19394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":9697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7297,"num_write_kb":29188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4221,"num_bytes_recovered":17281024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":166},{"shards":"2","objects":9529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9506","reported_seq":9624,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.731402+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:02.731402+0000","last_peered":"2024-09-05T13:16:02.731402+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:02.731402+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9506,"log_dups_size":0,"ondisk_log_size":9506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7015,"num_write_kb":28060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5113}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9610","reported_seq":9746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.321966+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:02.321966+0000","last_peered":"2024-09-05T13:16:02.321966+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:02.321966+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9610,"log_dups_size":0,"ondisk_log_size":9610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354368,"num_objects":9608,"num_object_clones":0,"num_object_copies":19216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5761,"num_write_kb":23044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5189},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9714","reported_seq":9897,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.360155+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:02.360155+0000","last_peered":"2024-09-05T13:16:02.360155+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:02.360155+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9714,"log_dups_size":0,"ondisk_log_size":9714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39755776,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5613,"num_write_kb":22452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9723","reported_seq":9855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.320115+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:02.320115+0000","last_peered":"2024-09-05T13:16:02.320115+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:02.320115+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9723,"log_dups_size":0,"ondisk_log_size":9723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39809024,"num_objects":9719,"num_object_clones":0,"num_object_copies":19438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5853,"num_write_kb":23412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5153},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9782","reported_seq":9916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:02.362556+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:02.362556+0000","last_peered":"2024-09-05T13:16:02.362556+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:02.362556+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9782,"log_dups_size":0,"ondisk_log_size":9782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6450,"num_write_kb":25800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5206},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633614819,"num_objects":154692,"num_object_clones":0,"num_object_copies":309384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49108,"num_objects_misplaced":166,"num_objects_unfound":0,"num_objects_dirty":154692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110491,"num_write_kb":441964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4224,"num_bytes_recovered":17293312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1289113600,"data_stored":1289030501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154773,"ondisk_log_size":154773,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704972,"kb_used_data":452540,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93666868,"statfs":{"total":96636764160,"available":95914872832,"internally_reserved":0,"allocated":463400960,"data_stored":463155870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":258469549},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":855276,"kb_used_data":534876,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93516564,"statfs":{"total":96636764160,"available":95760961536,"internally_reserved":0,"allocated":547713024,"data_stored":547463857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509876,"kb_used_data":271332,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93861964,"statfs":{"total":96636764160,"available":96114651136,"internally_reserved":0,"allocated":277843968,"data_stored":277522541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547807232,"data_stored":547803619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":278007808,"data_stored":277931935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463298560,"data_stored":463294947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:07.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 214 20 156732 156712 2.85981 1.6875 0.0296192 0.0273108 2024-09-05T13:16:08.243 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 215 20 157176 157156 2.85454 1.73438 0.0474513 0.0273604 2024-09-05T13:16:09.243 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 216 20 157541 157521 2.84792 1.42578 0.0104992 0.027378 2024-09-05T13:16:09.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:10.243 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 217 20 157969 157949 2.8425 1.67188 0.0149823 0.0274762 2024-09-05T13:16:10.360 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:10.363 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:10.392 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-05T13:16:09.592240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642462198,"num_objects":156854,"num_object_clones":0,"num_object_copies":313708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112653,"num_write_kb":450606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4393,"num_bytes_recovered":17981459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156936,"ondisk_log_size":156936,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2089168,"kb_used_data":1269472,"kb_used_omap":12,"kb_used_meta":819635,"kb_avail":281026352,"statfs":{"total":289910292480,"available":287770984448,"internally_reserved":0,"allocated":1299939328,"data_stored":1299123644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":839306246},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,4,7,8,8,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17891328,"num_objects":4368,"num_object_clones":0,"num_object_copies":8736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-308,"num_objects_unfound":0,"num_objects_dirty":4368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4368,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003324"},"pg_stats":[{"pgid":"2.f","version":"43'9773","reported_seq":9843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.334261+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:07.334261+0000","last_peered":"2024-09-05T13:16:07.334261+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:07.334261+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9773,"log_dups_size":0,"ondisk_log_size":9773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40030208,"num_objects":9773,"num_object_clones":0,"num_object_copies":19546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9773,"num_write_kb":39092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5352},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9719","reported_seq":9819,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.793366+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:07.793366+0000","last_peered":"2024-09-05T13:16:07.793366+0000","last_clean":"2024-09-05T13:16:07.793366+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:07.793366+0000","last_undegraded":"2024-09-05T13:16:07.793366+0000","last_fullsized":"2024-09-05T13:16:07.793366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9719,"log_dups_size":0,"ondisk_log_size":9719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39784448,"num_objects":9713,"num_object_clones":0,"num_object_copies":19426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7729,"num_write_kb":30916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9933","reported_seq":10056,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.710498+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:07.710498+0000","last_peered":"2024-09-05T13:16:07.710498+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:07.710498+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669184,"num_objects":9929,"num_object_clones":0,"num_object_copies":19858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7744,"num_write_kb":30976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5437}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9841","reported_seq":9906,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.799322+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:07.799322+0000","last_peered":"2024-09-05T13:16:07.799322+0000","last_clean":"2024-09-05T13:16:07.799322+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:07.799322+0000","last_undegraded":"2024-09-05T13:16:07.799322+0000","last_fullsized":"2024-09-05T13:16:07.799322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9841,"log_dups_size":0,"ondisk_log_size":9841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40308736,"num_objects":9841,"num_object_clones":0,"num_object_copies":19682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9841,"num_write_kb":39364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9740","reported_seq":9869,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.369157+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:07.369157+0000","last_peered":"2024-09-05T13:16:07.369157+0000","last_clean":"2024-09-05T13:16:07.369157+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:07.369157+0000","last_undegraded":"2024-09-05T13:16:07.369157+0000","last_fullsized":"2024-09-05T13:16:07.369157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39866368,"num_objects":9733,"num_object_clones":0,"num_object_copies":19466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6280,"num_write_kb":25120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9899","reported_seq":10033,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.382653+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:07.382653+0000","last_peered":"2024-09-05T13:16:07.382653+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:07.382653+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9899,"log_dups_size":0,"ondisk_log_size":9899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40525824,"num_objects":9894,"num_object_clones":0,"num_object_copies":19788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6354,"num_write_kb":25416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5443},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9660","reported_seq":9795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.359487+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:16:07.359487+0000","last_peered":"2024-09-05T13:16:07.359487+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:07.359487+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9660,"log_dups_size":0,"ondisk_log_size":9660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39542784,"num_objects":9654,"num_object_clones":0,"num_object_copies":19308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5979,"num_write_kb":23916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9907","reported_seq":10034,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.306375+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:07.306375+0000","last_peered":"2024-09-05T13:16:07.306375+0000","last_clean":"2024-09-05T13:16:07.306375+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:07.306375+0000","last_undegraded":"2024-09-05T13:16:07.306375+0000","last_fullsized":"2024-09-05T13:16:07.306375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9907,"log_dups_size":0,"ondisk_log_size":9907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40570880,"num_objects":9905,"num_object_clones":0,"num_object_copies":19810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6517,"num_write_kb":26068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9839","reported_seq":9969,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.677459+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:07.677459+0000","last_peered":"2024-09-05T13:16:07.677459+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:07.677459+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40271872,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7493,"num_write_kb":29972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5381}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9789","reported_seq":9922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.226100+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:07.226100+0000","last_peered":"2024-09-05T13:16:07.226100+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:07.226100+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9789,"log_dups_size":0,"ondisk_log_size":9789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40062976,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5327},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9835","reported_seq":18726,"reported_epoch":43,"state":"active+undersized+remapped","last_fresh":"2024-09-05T13:16:07.629804+0000","last_change":"2024-09-05T13:16:07.629804+0000","last_active":"2024-09-05T13:16:07.629804+0000","last_peered":"2024-09-05T13:16:07.629804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:16:07.629804+0000","last_undegraded":"2024-09-05T13:16:07.629804+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40263680,"num_objects":9830,"num_object_clones":0,"num_object_copies":19660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":29720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":9830}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9636","reported_seq":9754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.775207+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:07.775207+0000","last_peered":"2024-09-05T13:16:07.775207+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:07.775207+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9636,"log_dups_size":0,"ondisk_log_size":9636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39444480,"num_objects":9630,"num_object_clones":0,"num_object_copies":19260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7145,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5243}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9752","reported_seq":9885,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.337004+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:07.337004+0000","last_peered":"2024-09-05T13:16:07.337004+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:07.337004+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9752,"log_dups_size":0,"ondisk_log_size":9752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39936000,"num_objects":9750,"num_object_clones":0,"num_object_copies":19500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5903,"num_write_kb":23612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5331},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9846","reported_seq":10029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.284928+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:07.284928+0000","last_peered":"2024-09-05T13:16:07.284928+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:07.284928+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9846,"log_dups_size":0,"ondisk_log_size":9846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40296448,"num_objects":9838,"num_object_clones":0,"num_object_copies":19676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5745,"num_write_kb":22980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5394},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9846","reported_seq":9980,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.369204+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:07.369204+0000","last_peered":"2024-09-05T13:16:07.369204+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:07.369204+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9846,"log_dups_size":0,"ondisk_log_size":9846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40312832,"num_objects":9842,"num_object_clones":0,"num_object_copies":19684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5276},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9918","reported_seq":10053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.401092+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:07.401092+0000","last_peered":"2024-09-05T13:16:07.401092+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:07.401092+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9918,"log_dups_size":0,"ondisk_log_size":9918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40575459,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642462179,"num_objects":156852,"num_object_clones":0,"num_object_copies":313704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112651,"num_write_kb":450604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4392,"num_bytes_recovered":17981440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300123648,"data_stored":1300040549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156933,"ondisk_log_size":156933,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713440,"kb_used_data":456848,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93658400,"statfs":{"total":96636764160,"available":95906201600,"internally_reserved":0,"allocated":467812352,"data_stored":467567262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865852,"kb_used_data":541292,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93505988,"statfs":{"total":96636764160,"available":95750131712,"internally_reserved":0,"allocated":554283008,"data_stored":554033841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,1,3,3,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509876,"kb_used_data":271332,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93861964,"statfs":{"total":96636764160,"available":96114651136,"internally_reserved":0,"allocated":277843968,"data_stored":277522541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554336256,"data_stored":554332643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":278007808,"data_stored":277931935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467779584,"data_stored":467775971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:10.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:11.017 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:11.018 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:11.053 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-05T13:16:09.592240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642462198,"num_objects":156854,"num_object_clones":0,"num_object_copies":313708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112653,"num_write_kb":450606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4393,"num_bytes_recovered":17981459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156936,"ondisk_log_size":156936,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2089168,"kb_used_data":1269472,"kb_used_omap":12,"kb_used_meta":819635,"kb_avail":281026352,"statfs":{"total":289910292480,"available":287770984448,"internally_reserved":0,"allocated":1299939328,"data_stored":1299123644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":839306246},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,4,7,8,8,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17891328,"num_objects":4368,"num_object_clones":0,"num_object_copies":8736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-308,"num_objects_unfound":0,"num_objects_dirty":4368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4368,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003324"},"pg_stats":[{"pgid":"2.f","version":"43'9773","reported_seq":9843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.334261+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:07.334261+0000","last_peered":"2024-09-05T13:16:07.334261+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:07.334261+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9773,"log_dups_size":0,"ondisk_log_size":9773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40030208,"num_objects":9773,"num_object_clones":0,"num_object_copies":19546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9773,"num_write_kb":39092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5352},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9719","reported_seq":9819,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.793366+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:07.793366+0000","last_peered":"2024-09-05T13:16:07.793366+0000","last_clean":"2024-09-05T13:16:07.793366+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:07.793366+0000","last_undegraded":"2024-09-05T13:16:07.793366+0000","last_fullsized":"2024-09-05T13:16:07.793366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9719,"log_dups_size":0,"ondisk_log_size":9719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39784448,"num_objects":9713,"num_object_clones":0,"num_object_copies":19426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7729,"num_write_kb":30916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9933","reported_seq":10056,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.710498+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:07.710498+0000","last_peered":"2024-09-05T13:16:07.710498+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:07.710498+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669184,"num_objects":9929,"num_object_clones":0,"num_object_copies":19858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7744,"num_write_kb":30976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5437}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9841","reported_seq":9906,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.799322+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:07.799322+0000","last_peered":"2024-09-05T13:16:07.799322+0000","last_clean":"2024-09-05T13:16:07.799322+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:07.799322+0000","last_undegraded":"2024-09-05T13:16:07.799322+0000","last_fullsized":"2024-09-05T13:16:07.799322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9841,"log_dups_size":0,"ondisk_log_size":9841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40308736,"num_objects":9841,"num_object_clones":0,"num_object_copies":19682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9841,"num_write_kb":39364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9740","reported_seq":9869,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.369157+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:07.369157+0000","last_peered":"2024-09-05T13:16:07.369157+0000","last_clean":"2024-09-05T13:16:07.369157+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:07.369157+0000","last_undegraded":"2024-09-05T13:16:07.369157+0000","last_fullsized":"2024-09-05T13:16:07.369157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39866368,"num_objects":9733,"num_object_clones":0,"num_object_copies":19466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6280,"num_write_kb":25120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9899","reported_seq":10033,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.382653+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:07.382653+0000","last_peered":"2024-09-05T13:16:07.382653+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:07.382653+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9899,"log_dups_size":0,"ondisk_log_size":9899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40525824,"num_objects":9894,"num_object_clones":0,"num_object_copies":19788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6354,"num_write_kb":25416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5443},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9660","reported_seq":9795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.359487+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:16:07.359487+0000","last_peered":"2024-09-05T13:16:07.359487+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:07.359487+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9660,"log_dups_size":0,"ondisk_log_size":9660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39542784,"num_objects":9654,"num_object_clones":0,"num_object_copies":19308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5979,"num_write_kb":23916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9907","reported_seq":10034,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.306375+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:07.306375+0000","last_peered":"2024-09-05T13:16:07.306375+0000","last_clean":"2024-09-05T13:16:07.306375+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:07.306375+0000","last_undegraded":"2024-09-05T13:16:07.306375+0000","last_fullsized":"2024-09-05T13:16:07.306375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9907,"log_dups_size":0,"ondisk_log_size":9907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40570880,"num_objects":9905,"num_object_clones":0,"num_object_copies":19810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6517,"num_write_kb":26068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.914321+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:13:17.914321+0000","last_peered":"2024-09-05T13:13:17.914321+0000","last_clean":"2024-09-05T13:13:17.914321+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:13:17.914321+0000","last_undegraded":"2024-09-05T13:13:17.914321+0000","last_fullsized":"2024-09-05T13:13:17.914321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9839","reported_seq":9969,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.677459+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:07.677459+0000","last_peered":"2024-09-05T13:16:07.677459+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:07.677459+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40271872,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7493,"num_write_kb":29972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5381}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9789","reported_seq":9922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.226100+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:07.226100+0000","last_peered":"2024-09-05T13:16:07.226100+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:07.226100+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9789,"log_dups_size":0,"ondisk_log_size":9789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40062976,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5327},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9835","reported_seq":18726,"reported_epoch":43,"state":"active+undersized+remapped","last_fresh":"2024-09-05T13:16:07.629804+0000","last_change":"2024-09-05T13:16:07.629804+0000","last_active":"2024-09-05T13:16:07.629804+0000","last_peered":"2024-09-05T13:16:07.629804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:16:07.629804+0000","last_undegraded":"2024-09-05T13:16:07.629804+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40263680,"num_objects":9830,"num_object_clones":0,"num_object_copies":19660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":29720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":9830}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.451973+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:13:18.451973+0000","last_peered":"2024-09-05T13:13:18.451973+0000","last_clean":"2024-09-05T13:13:18.451973+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:13:18.451973+0000","last_undegraded":"2024-09-05T13:13:18.451973+0000","last_fullsized":"2024-09-05T13:13:18.451973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9636","reported_seq":9754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.775207+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:07.775207+0000","last_peered":"2024-09-05T13:16:07.775207+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:07.775207+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9636,"log_dups_size":0,"ondisk_log_size":9636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39444480,"num_objects":9630,"num_object_clones":0,"num_object_copies":19260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7145,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5243}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9752","reported_seq":9885,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.337004+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:07.337004+0000","last_peered":"2024-09-05T13:16:07.337004+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:07.337004+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9752,"log_dups_size":0,"ondisk_log_size":9752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39936000,"num_objects":9750,"num_object_clones":0,"num_object_copies":19500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5903,"num_write_kb":23612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5331},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.116185+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:13:18.116185+0000","last_peered":"2024-09-05T13:13:18.116185+0000","last_clean":"2024-09-05T13:13:18.116185+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:13:18.116185+0000","last_undegraded":"2024-09-05T13:13:18.116185+0000","last_fullsized":"2024-09-05T13:13:18.116185+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9846","reported_seq":10029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.284928+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:07.284928+0000","last_peered":"2024-09-05T13:16:07.284928+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:07.284928+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9846,"log_dups_size":0,"ondisk_log_size":9846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40296448,"num_objects":9838,"num_object_clones":0,"num_object_copies":19676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5745,"num_write_kb":22980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5394},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9846","reported_seq":9980,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.369204+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:07.369204+0000","last_peered":"2024-09-05T13:16:07.369204+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:07.369204+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9846,"log_dups_size":0,"ondisk_log_size":9846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40312832,"num_objects":9842,"num_object_clones":0,"num_object_copies":19684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5276},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.478450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:13:18.478450+0000","last_peered":"2024-09-05T13:13:18.478450+0000","last_clean":"2024-09-05T13:13:18.478450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:13:18.478450+0000","last_undegraded":"2024-09-05T13:13:18.478450+0000","last_fullsized":"2024-09-05T13:13:18.478450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.908009+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:13:17.908009+0000","last_peered":"2024-09-05T13:13:17.908009+0000","last_clean":"2024-09-05T13:13:17.908009+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:13:17.908009+0000","last_undegraded":"2024-09-05T13:13:17.908009+0000","last_fullsized":"2024-09-05T13:13:17.908009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9918","reported_seq":10053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.401092+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:07.401092+0000","last_peered":"2024-09-05T13:16:07.401092+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:07.401092+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9918,"log_dups_size":0,"ondisk_log_size":9918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40575459,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642462179,"num_objects":156852,"num_object_clones":0,"num_object_copies":313704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112651,"num_write_kb":450604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4392,"num_bytes_recovered":17981440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300123648,"data_stored":1300040549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156933,"ondisk_log_size":156933,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713440,"kb_used_data":456848,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93658400,"statfs":{"total":96636764160,"available":95906201600,"internally_reserved":0,"allocated":467812352,"data_stored":467567262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865852,"kb_used_data":541292,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93505988,"statfs":{"total":96636764160,"available":95750131712,"internally_reserved":0,"allocated":554283008,"data_stored":554033841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,1,3,3,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509876,"kb_used_data":271332,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93861964,"statfs":{"total":96636764160,"available":96114651136,"internally_reserved":0,"allocated":277843968,"data_stored":277522541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554336256,"data_stored":554332643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":278007808,"data_stored":277931935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467779584,"data_stored":467775971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:11.054 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'9773", 'reported_seq': 9843, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.334261+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:07.334261+0000', 'last_peered': '2024-09-05T13:16:07.334261+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:07.334261+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 9773, 'log_dups_size': 0, 'ondisk_log_size': 9773, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40030208, 'num_objects': 9773, 'num_object_clones': 0, 'num_object_copies': 19546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9773, 'num_write_kb': 39092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5352}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'9933", 'reported_seq': 10056, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.710498+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:07.710498+0000', 'last_peered': '2024-09-05T13:16:07.710498+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:07.710498+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 9933, 'log_dups_size': 0, 'ondisk_log_size': 9933, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40669184, 'num_objects': 9929, 'num_object_clones': 0, 'num_object_copies': 19858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7744, 'num_write_kb': 30976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 5437}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'9899", 'reported_seq': 10033, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.382653+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:07.382653+0000', 'last_peered': '2024-09-05T13:16:07.382653+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:07.382653+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 9899, 'log_dups_size': 0, 'ondisk_log_size': 9899, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40525824, 'num_objects': 9894, 'num_object_clones': 0, 'num_object_copies': 19788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6354, 'num_write_kb': 25416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5443}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'9660", 'reported_seq': 9795, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.359487+0000', 'last_change': '2024-09-05T13:13:18.451326+0000', 'last_active': '2024-09-05T13:16:07.359487+0000', 'last_peered': '2024-09-05T13:16:07.359487+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:07.359487+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 9660, 'log_dups_size': 0, 'ondisk_log_size': 9660, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39542784, 'num_objects': 9654, 'num_object_clones': 0, 'num_object_copies': 19308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9654, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5979, 'num_write_kb': 23916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5218}, {'shards': '0,2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:11.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'9839", 'reported_seq': 9969, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.677459+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:07.677459+0000', 'last_peered': '2024-09-05T13:16:07.677459+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:07.677459+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 9839, 'log_dups_size': 0, 'ondisk_log_size': 9839, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40271872, 'num_objects': 9832, 'num_object_clones': 0, 'num_object_copies': 19664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9832, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7493, 'num_write_kb': 29972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 5381}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'9789", 'reported_seq': 9922, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.226100+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:07.226100+0000', 'last_peered': '2024-09-05T13:16:07.226100+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:07.226100+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 9789, 'log_dups_size': 0, 'ondisk_log_size': 9789, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40062976, 'num_objects': 9781, 'num_object_clones': 0, 'num_object_copies': 19562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9781, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6156, 'num_write_kb': 24624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5327}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'9835", 'reported_seq': 18726, 'reported_epoch': 43, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-05T13:16:07.629804+0000', 'last_change': '2024-09-05T13:16:07.629804+0000', 'last_active': '2024-09-05T13:16:07.629804+0000', 'last_peered': '2024-09-05T13:16:07.629804+0000', 'last_clean': '2024-09-05T13:13:15.895164+0000', 'last_became_active': '2024-09-05T13:13:18.447155+0000', 'last_became_peered': '2024-09-05T13:13:18.447155+0000', 'last_unstale': '2024-09-05T13:16:07.629804+0000', 'last_undegraded': '2024-09-05T13:16:07.629804+0000', 'last_fullsized': '2024-09-05T13:13:17.907423+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2441", 'last_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:50.064199+0000', 'objects_scrubbed': 2400, 'log_size': 9835, 'log_dups_size': 0, 'ondisk_log_size': 9835, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 873, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40263680, 'num_objects': 9830, 'num_object_clones': 0, 'num_object_copies': 19660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9830, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7430, 'num_write_kb': 29720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4389, 'num_bytes_recovered': 17969152, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '2', 'objects': 9830}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'9636", 'reported_seq': 9754, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.775207+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:07.775207+0000', 'last_peered': '2024-09-05T13:16:07.775207+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:07.775207+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 9636, 'log_dups_size': 0, 'ondisk_log_size': 9636, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39444480, 'num_objects': 9630, 'num_object_clones': 0, 'num_object_copies': 19260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7145, 'num_write_kb': 28580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 5243}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'9752", 'reported_seq': 9885, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.337004+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:07.337004+0000', 'last_peered': '2024-09-05T13:16:07.337004+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:07.337004+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 9752, 'log_dups_size': 0, 'ondisk_log_size': 9752, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39936000, 'num_objects': 9750, 'num_object_clones': 0, 'num_object_copies': 19500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9750, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5903, 'num_write_kb': 23612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5331}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:11.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'9846", 'reported_seq': 10029, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.284928+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:07.284928+0000', 'last_peered': '2024-09-05T13:16:07.284928+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:07.284928+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 9846, 'log_dups_size': 0, 'ondisk_log_size': 9846, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40296448, 'num_objects': 9838, 'num_object_clones': 0, 'num_object_copies': 19676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9838, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5745, 'num_write_kb': 22980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5394}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:11.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:11.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'9846", 'reported_seq': 9980, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.369204+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:07.369204+0000', 'last_peered': '2024-09-05T13:16:07.369204+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:07.369204+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 9846, 'log_dups_size': 0, 'ondisk_log_size': 9846, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40312832, 'num_objects': 9842, 'num_object_clones': 0, 'num_object_copies': 19684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5976, 'num_write_kb': 23904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5276}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:11.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:11.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'9918", 'reported_seq': 10053, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:07.401092+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:07.401092+0000', 'last_peered': '2024-09-05T13:16:07.401092+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:07.401092+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 9918, 'log_dups_size': 0, 'ondisk_log_size': 9918, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40575459, 'num_objects': 9907, 'num_object_clones': 0, 'num_object_copies': 19814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6586, 'num_write_kb': 26344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5342}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:11.057 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:11.243 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 218 20 158374 158354 2.83672 1.58203 0.0231857 0.0275327 2024-09-05T13:16:11.764 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:11.764 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 49106/313708 objects degraded (15.653%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":298,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156854,"data_bytes":642462198,"bytes_used":2139308032,"bytes_avail":287770984448,"bytes_total":289910292480,"degraded_objects":49106,"degraded_total":313708,"degraded_ratio":0.15653410177617402,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":158654,"recovering_keys_per_sec":0,"num_objects_recovered":310,"num_bytes_recovered":1269760,"num_keys_recovered":0,"write_bytes_sec":2235487,"read_op_per_sec":0,"write_op_per_sec":545},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:11.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:11.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:12.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 219 20 158773 158753 2.83089 1.55859 0.0472432 0.0275877 2024-09-05T13:16:12.354 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:12.355 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:12.385 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-05T13:16:11.593023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642462198,"num_objects":156854,"num_object_clones":0,"num_object_copies":313708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112653,"num_write_kb":450606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4393,"num_bytes_recovered":17981459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156936,"ondisk_log_size":156936,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087696,"kb_used_data":1276312,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281027824,"statfs":{"total":289910292480,"available":287772491776,"internally_reserved":0,"allocated":1306943488,"data_stored":1306117409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":830786566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,3,9,9,8,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8847360,"num_objects":2160,"num_object_clones":0,"num_object_copies":4320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-166,"num_objects_unfound":0,"num_objects_dirty":2160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2160,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003280"},"pg_stats":[{"pgid":"2.f","version":"43'9773","reported_seq":9843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.334261+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:07.334261+0000","last_peered":"2024-09-05T13:16:07.334261+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:07.334261+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9773,"log_dups_size":0,"ondisk_log_size":9773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40030208,"num_objects":9773,"num_object_clones":0,"num_object_copies":19546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9773,"num_write_kb":39092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5352},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'9719","reported_seq":9819,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.793366+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:07.793366+0000","last_peered":"2024-09-05T13:16:07.793366+0000","last_clean":"2024-09-05T13:16:07.793366+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:07.793366+0000","last_undegraded":"2024-09-05T13:16:07.793366+0000","last_fullsized":"2024-09-05T13:16:07.793366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9719,"log_dups_size":0,"ondisk_log_size":9719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39784448,"num_objects":9713,"num_object_clones":0,"num_object_copies":19426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7729,"num_write_kb":30916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'9933","reported_seq":10056,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.710498+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:07.710498+0000","last_peered":"2024-09-05T13:16:07.710498+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:07.710498+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669184,"num_objects":9929,"num_object_clones":0,"num_object_copies":19858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7744,"num_write_kb":30976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5437}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'9841","reported_seq":9906,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.799322+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:07.799322+0000","last_peered":"2024-09-05T13:16:07.799322+0000","last_clean":"2024-09-05T13:16:07.799322+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:07.799322+0000","last_undegraded":"2024-09-05T13:16:07.799322+0000","last_fullsized":"2024-09-05T13:16:07.799322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9841,"log_dups_size":0,"ondisk_log_size":9841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40308736,"num_objects":9841,"num_object_clones":0,"num_object_copies":19682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9841,"num_write_kb":39364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'9740","reported_seq":9869,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.369157+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:07.369157+0000","last_peered":"2024-09-05T13:16:07.369157+0000","last_clean":"2024-09-05T13:16:07.369157+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:07.369157+0000","last_undegraded":"2024-09-05T13:16:07.369157+0000","last_fullsized":"2024-09-05T13:16:07.369157+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39866368,"num_objects":9733,"num_object_clones":0,"num_object_copies":19466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6280,"num_write_kb":25120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'9899","reported_seq":10033,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.382653+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:07.382653+0000","last_peered":"2024-09-05T13:16:07.382653+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:07.382653+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":9899,"log_dups_size":0,"ondisk_log_size":9899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40525824,"num_objects":9894,"num_object_clones":0,"num_object_copies":19788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6354,"num_write_kb":25416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5443},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'9660","reported_seq":9795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.359487+0000","last_change":"2024-09-05T13:13:18.451326+0000","last_active":"2024-09-05T13:16:07.359487+0000","last_peered":"2024-09-05T13:16:07.359487+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:07.359487+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9660,"log_dups_size":0,"ondisk_log_size":9660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39542784,"num_objects":9654,"num_object_clones":0,"num_object_copies":19308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5979,"num_write_kb":23916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":4435}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'9907","reported_seq":10034,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:16:07.306375+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:07.306375+0000","last_peered":"2024-09-05T13:16:07.306375+0000","last_clean":"2024-09-05T13:16:07.306375+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:07.306375+0000","last_undegraded":"2024-09-05T13:16:07.306375+0000","last_fullsized":"2024-09-05T13:16:07.306375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":9907,"log_dups_size":0,"ondisk_log_size":9907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40570880,"num_objects":9905,"num_object_clones":0,"num_object_copies":19810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6517,"num_write_kb":26068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'9839","reported_seq":9969,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.677459+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:07.677459+0000","last_peered":"2024-09-05T13:16:07.677459+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:07.677459+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40271872,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7493,"num_write_kb":29972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5381}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.957099+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:13:17.957099+0000","last_peered":"2024-09-05T13:13:17.957099+0000","last_clean":"2024-09-05T13:13:17.957099+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:13:17.957099+0000","last_undegraded":"2024-09-05T13:13:17.957099+0000","last_fullsized":"2024-09-05T13:13:17.957099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'9789","reported_seq":9922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.226100+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:07.226100+0000","last_peered":"2024-09-05T13:16:07.226100+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:07.226100+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9789,"log_dups_size":0,"ondisk_log_size":9789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40062976,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5327},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:18.020211+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:13:18.020211+0000","last_peered":"2024-09-05T13:13:18.020211+0000","last_clean":"2024-09-05T13:13:18.020211+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:13:18.020211+0000","last_undegraded":"2024-09-05T13:13:18.020211+0000","last_fullsized":"2024-09-05T13:13:18.020211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'9835","reported_seq":18726,"reported_epoch":43,"state":"active+undersized+remapped","last_fresh":"2024-09-05T13:16:07.629804+0000","last_change":"2024-09-05T13:16:07.629804+0000","last_active":"2024-09-05T13:16:07.629804+0000","last_peered":"2024-09-05T13:16:07.629804+0000","last_clean":"2024-09-05T13:13:15.895164+0000","last_became_active":"2024-09-05T13:13:18.447155+0000","last_became_peered":"2024-09-05T13:13:18.447155+0000","last_unstale":"2024-09-05T13:16:07.629804+0000","last_undegraded":"2024-09-05T13:16:07.629804+0000","last_fullsized":"2024-09-05T13:13:17.907423+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40263680,"num_objects":9830,"num_object_clones":0,"num_object_copies":19660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":29720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":9830}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'9636","reported_seq":9754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.775207+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:07.775207+0000","last_peered":"2024-09-05T13:16:07.775207+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:07.775207+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9636,"log_dups_size":0,"ondisk_log_size":9636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39444480,"num_objects":9630,"num_object_clones":0,"num_object_copies":19260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7145,"num_write_kb":28580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5243}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'9752","reported_seq":9885,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.337004+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:07.337004+0000","last_peered":"2024-09-05T13:16:07.337004+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:07.337004+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9752,"log_dups_size":0,"ondisk_log_size":9752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39936000,"num_objects":9750,"num_object_clones":0,"num_object_copies":19500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5903,"num_write_kb":23612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5331},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'9846","reported_seq":10029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.284928+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:07.284928+0000","last_peered":"2024-09-05T13:16:07.284928+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:07.284928+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9846,"log_dups_size":0,"ondisk_log_size":9846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40296448,"num_objects":9838,"num_object_clones":0,"num_object_copies":19676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5745,"num_write_kb":22980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5394},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-05T13:13:17.945463+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:13:17.945463+0000","last_peered":"2024-09-05T13:13:17.945463+0000","last_clean":"2024-09-05T13:13:17.945463+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:13:17.945463+0000","last_undegraded":"2024-09-05T13:13:17.945463+0000","last_fullsized":"2024-09-05T13:13:17.945463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'9846","reported_seq":9980,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.369204+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:07.369204+0000","last_peered":"2024-09-05T13:16:07.369204+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:07.369204+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9846,"log_dups_size":0,"ondisk_log_size":9846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40312832,"num_objects":9842,"num_object_clones":0,"num_object_copies":19684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5276},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'9918","reported_seq":10053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:07.401092+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:07.401092+0000","last_peered":"2024-09-05T13:16:07.401092+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:07.401092+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":9918,"log_dups_size":0,"ondisk_log_size":9918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40575459,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642462179,"num_objects":156852,"num_object_clones":0,"num_object_copies":313704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112651,"num_write_kb":450604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4392,"num_bytes_recovered":17981440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1307156480,"data_stored":1307073381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156933,"ondisk_log_size":156933,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713440,"kb_used_data":456848,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93658400,"statfs":{"total":96636764160,"available":95906201600,"internally_reserved":0,"allocated":467812352,"data_stored":467567262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":262729389},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865852,"kb_used_data":541292,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93505988,"statfs":{"total":96636764160,"available":95750131712,"internally_reserved":0,"allocated":554283008,"data_stored":554033841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,1,3,3,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508404,"kb_used_data":278172,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93863436,"statfs":{"total":96636764160,"available":96116158464,"internally_reserved":0,"allocated":284848128,"data_stored":284516306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235728559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554336256,"data_stored":554332643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285040640,"data_stored":284964767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467779584,"data_stored":467775971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:13.243 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:2024-09-05T13:16:13.243266+0000 min lat: 0.00201413 max lat: 2.69414 avg lat: 0.0276372 2024-09-05T13:16:13.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-05T13:16:13.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 220 20 159214 159194 2.82585 1.72266 0.111605 0.0276372 2024-09-05T13:16:14.243 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 221 20 159659 159639 2.82093 1.73828 0.0214264 0.0276865 2024-09-05T13:16:15.243 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 222 20 160112 160092 2.81619 1.76953 0.0367998 0.027734 2024-09-05T13:16:15.388 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:15.960 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:15.960 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:15.990 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-05T13:16:15.594182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650912246,"num_objects":158917,"num_object_clones":0,"num_object_copies":317834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4341,"num_objects_unfound":0,"num_objects_dirty":158917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114716,"num_write_kb":458858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4487,"num_bytes_recovered":18366483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158999,"ondisk_log_size":158999,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2146556,"kb_used_data":1286596,"kb_used_omap":12,"kb_used_meta":859891,"kb_avail":280968964,"statfs":{"total":289910292480,"available":287712219136,"internally_reserved":0,"allocated":1317474304,"data_stored":1316627435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":880528390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,9,12,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11829248,"num_objects":2888,"num_object_clones":0,"num_object_copies":5776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4439,"num_objects_misplaced":4175,"num_objects_unfound":0,"num_objects_dirty":2888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2888,"num_write_kb":11552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002590"},"pg_stats":[{"pgid":"2.f","version":"45'9915","reported_seq":9989,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.223049+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:12.223049+0000","last_peered":"2024-09-05T13:16:12.223049+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:12.223049+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9915,"log_dups_size":0,"ondisk_log_size":9915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40611840,"num_objects":9915,"num_object_clones":0,"num_object_copies":19830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":39660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5494},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9828","reported_seq":9932,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.770401+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:12.770401+0000","last_peered":"2024-09-05T13:16:12.770401+0000","last_clean":"2024-09-05T13:16:12.770401+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:12.770401+0000","last_undegraded":"2024-09-05T13:16:12.770401+0000","last_fullsized":"2024-09-05T13:16:12.770401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9828,"log_dups_size":0,"ondisk_log_size":9828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40230912,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7838,"num_write_kb":31352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10055","reported_seq":10182,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.763633+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:12.763633+0000","last_peered":"2024-09-05T13:16:12.763633+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:12.763633+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9963","reported_seq":10032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.744415+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:12.744415+0000","last_peered":"2024-09-05T13:16:12.744415+0000","last_clean":"2024-09-05T13:16:12.744415+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:12.744415+0000","last_undegraded":"2024-09-05T13:16:12.744415+0000","last_fullsized":"2024-09-05T13:16:12.744415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9963,"log_dups_size":0,"ondisk_log_size":9963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40808448,"num_objects":9963,"num_object_clones":0,"num_object_copies":19926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9963,"num_write_kb":39852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9877","reported_seq":10011,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.419398+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:12.419398+0000","last_peered":"2024-09-05T13:16:12.419398+0000","last_clean":"2024-09-05T13:16:12.419398+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:12.419398+0000","last_undegraded":"2024-09-05T13:16:12.419398+0000","last_fullsized":"2024-09-05T13:16:12.419398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9877,"log_dups_size":0,"ondisk_log_size":9877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10034","reported_seq":10172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.400249+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:12.400249+0000","last_peered":"2024-09-05T13:16:12.400249+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:12.400249+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41078784,"num_objects":10029,"num_object_clones":0,"num_object_copies":20058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5578},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9782","reported_seq":10110,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.379216+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:12.379216+0000","last_peered":"2024-09-05T13:16:12.379216+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:12.379216+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9782,"log_dups_size":0,"ondisk_log_size":9782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40042496,"num_objects":9776,"num_object_clones":0,"num_object_copies":19552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4341,"num_objects_unfound":0,"num_objects_dirty":9776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6101,"num_write_kb":24404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":385024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5434},{"shards":"0,2","objects":4341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10042","reported_seq":10173,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.305430+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:12.305430+0000","last_peered":"2024-09-05T13:16:12.305430+0000","last_clean":"2024-09-05T13:16:12.305430+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:12.305430+0000","last_undegraded":"2024-09-05T13:16:12.305430+0000","last_fullsized":"2024-09-05T13:16:12.305430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41123840,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":26608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9968","reported_seq":10102,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.806785+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:12.806785+0000","last_peered":"2024-09-05T13:16:12.806785+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:12.806785+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800256,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7622,"num_write_kb":30488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9908","reported_seq":10045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.410476+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:12.410476+0000","last_peered":"2024-09-05T13:16:12.410476+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:12.410476+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9908,"log_dups_size":0,"ondisk_log_size":9908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6275,"num_write_kb":25100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5446},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9962","reported_seq":18861,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.825882+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:12.825882+0000","last_peered":"2024-09-05T13:16:12.825882+0000","last_clean":"2024-09-05T13:16:12.825882+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:12.825882+0000","last_undegraded":"2024-09-05T13:16:12.825882+0000","last_fullsized":"2024-09-05T13:16:12.825882+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9962,"log_dups_size":0,"ondisk_log_size":9962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40783872,"num_objects":9957,"num_object_clones":0,"num_object_copies":19914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7557,"num_write_kb":30228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9770","reported_seq":9892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.810355+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:12.810355+0000","last_peered":"2024-09-05T13:16:12.810355+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:12.810355+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9770,"log_dups_size":0,"ondisk_log_size":9770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7279,"num_write_kb":29116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5377}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9879","reported_seq":10019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.407377+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:12.407377+0000","last_peered":"2024-09-05T13:16:12.407377+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:12.407377+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40456192,"num_objects":9877,"num_object_clones":0,"num_object_copies":19754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6030,"num_write_kb":24120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5458},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9980","reported_seq":10167,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.334120+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:12.334120+0000","last_peered":"2024-09-05T13:16:12.334120+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:12.334120+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9980,"log_dups_size":0,"ondisk_log_size":9980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40845312,"num_objects":9972,"num_object_clones":0,"num_object_copies":19944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5879,"num_write_kb":23516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5528},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9982","reported_seq":10119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.424868+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:12.424868+0000","last_peered":"2024-09-05T13:16:12.424868+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:12.424868+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9982,"log_dups_size":0,"ondisk_log_size":9982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6112,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5412},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10051","reported_seq":10189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.324301+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:12.324301+0000","last_peered":"2024-09-05T13:16:12.324301+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:12.324301+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41120227,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6719,"num_write_kb":26876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650912227,"num_objects":158915,"num_object_clones":0,"num_object_copies":317830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4341,"num_objects_unfound":0,"num_objects_dirty":158915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114714,"num_write_kb":458856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4486,"num_bytes_recovered":18366464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1317683200,"data_stored":1317600101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158996,"ondisk_log_size":158996,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739664,"kb_used_data":460864,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93632176,"statfs":{"total":96636764160,"available":95879348224,"internally_reserved":0,"allocated":471924736,"data_stored":471669251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":285470381},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":898488,"kb_used_data":547560,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93473352,"statfs":{"total":96636764160,"available":95716712448,"internally_reserved":0,"allocated":560701440,"data_stored":560441878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,5,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508404,"kb_used_data":278172,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93863436,"statfs":{"total":96636764160,"available":96116158464,"internally_reserved":0,"allocated":284848128,"data_stored":284516306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235728559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":560689152,"data_stored":560685539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285040640,"data_stored":284964767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471953408,"data_stored":471949795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:15.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:16.243 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 223 20 160522 160502 2.81075 1.60156 0.0284144 0.027788 2024-09-05T13:16:16.548 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:16.551 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:16.584 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-05T13:16:15.594182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650912246,"num_objects":158917,"num_object_clones":0,"num_object_copies":317834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4341,"num_objects_unfound":0,"num_objects_dirty":158917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114716,"num_write_kb":458858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4487,"num_bytes_recovered":18366483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158999,"ondisk_log_size":158999,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2146556,"kb_used_data":1286596,"kb_used_omap":12,"kb_used_meta":859891,"kb_avail":280968964,"statfs":{"total":289910292480,"available":287712219136,"internally_reserved":0,"allocated":1317474304,"data_stored":1316627435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":880528390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,9,12,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11829248,"num_objects":2888,"num_object_clones":0,"num_object_copies":5776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4439,"num_objects_misplaced":4175,"num_objects_unfound":0,"num_objects_dirty":2888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2888,"num_write_kb":11552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002590"},"pg_stats":[{"pgid":"2.f","version":"45'9915","reported_seq":9989,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.223049+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:12.223049+0000","last_peered":"2024-09-05T13:16:12.223049+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:12.223049+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9915,"log_dups_size":0,"ondisk_log_size":9915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40611840,"num_objects":9915,"num_object_clones":0,"num_object_copies":19830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":39660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5494},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9828","reported_seq":9932,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.770401+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:12.770401+0000","last_peered":"2024-09-05T13:16:12.770401+0000","last_clean":"2024-09-05T13:16:12.770401+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:12.770401+0000","last_undegraded":"2024-09-05T13:16:12.770401+0000","last_fullsized":"2024-09-05T13:16:12.770401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9828,"log_dups_size":0,"ondisk_log_size":9828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40230912,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7838,"num_write_kb":31352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10055","reported_seq":10182,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.763633+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:12.763633+0000","last_peered":"2024-09-05T13:16:12.763633+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:12.763633+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9963","reported_seq":10032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.744415+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:12.744415+0000","last_peered":"2024-09-05T13:16:12.744415+0000","last_clean":"2024-09-05T13:16:12.744415+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:12.744415+0000","last_undegraded":"2024-09-05T13:16:12.744415+0000","last_fullsized":"2024-09-05T13:16:12.744415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9963,"log_dups_size":0,"ondisk_log_size":9963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40808448,"num_objects":9963,"num_object_clones":0,"num_object_copies":19926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9963,"num_write_kb":39852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9877","reported_seq":10011,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.419398+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:12.419398+0000","last_peered":"2024-09-05T13:16:12.419398+0000","last_clean":"2024-09-05T13:16:12.419398+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:12.419398+0000","last_undegraded":"2024-09-05T13:16:12.419398+0000","last_fullsized":"2024-09-05T13:16:12.419398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":9877,"log_dups_size":0,"ondisk_log_size":9877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10034","reported_seq":10172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.400249+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:12.400249+0000","last_peered":"2024-09-05T13:16:12.400249+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:12.400249+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41078784,"num_objects":10029,"num_object_clones":0,"num_object_copies":20058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5578},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9782","reported_seq":10110,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.379216+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:12.379216+0000","last_peered":"2024-09-05T13:16:12.379216+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:12.379216+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9782,"log_dups_size":0,"ondisk_log_size":9782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40042496,"num_objects":9776,"num_object_clones":0,"num_object_copies":19552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4341,"num_objects_unfound":0,"num_objects_dirty":9776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6101,"num_write_kb":24404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":385024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5434},{"shards":"0,2","objects":4341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10042","reported_seq":10173,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.305430+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:12.305430+0000","last_peered":"2024-09-05T13:16:12.305430+0000","last_clean":"2024-09-05T13:16:12.305430+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:12.305430+0000","last_undegraded":"2024-09-05T13:16:12.305430+0000","last_fullsized":"2024-09-05T13:16:12.305430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41123840,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":26608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9968","reported_seq":10102,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.806785+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:12.806785+0000","last_peered":"2024-09-05T13:16:12.806785+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:12.806785+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800256,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7622,"num_write_kb":30488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9908","reported_seq":10045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.410476+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:12.410476+0000","last_peered":"2024-09-05T13:16:12.410476+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:12.410476+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":9908,"log_dups_size":0,"ondisk_log_size":9908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6275,"num_write_kb":25100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5446},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9962","reported_seq":18861,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.825882+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:12.825882+0000","last_peered":"2024-09-05T13:16:12.825882+0000","last_clean":"2024-09-05T13:16:12.825882+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:12.825882+0000","last_undegraded":"2024-09-05T13:16:12.825882+0000","last_fullsized":"2024-09-05T13:16:12.825882+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9962,"log_dups_size":0,"ondisk_log_size":9962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40783872,"num_objects":9957,"num_object_clones":0,"num_object_copies":19914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7557,"num_write_kb":30228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9770","reported_seq":9892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.810355+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:12.810355+0000","last_peered":"2024-09-05T13:16:12.810355+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:12.810355+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9770,"log_dups_size":0,"ondisk_log_size":9770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7279,"num_write_kb":29116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5377}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9879","reported_seq":10019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.407377+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:12.407377+0000","last_peered":"2024-09-05T13:16:12.407377+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:12.407377+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40456192,"num_objects":9877,"num_object_clones":0,"num_object_copies":19754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6030,"num_write_kb":24120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5458},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9980","reported_seq":10167,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.334120+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:12.334120+0000","last_peered":"2024-09-05T13:16:12.334120+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:12.334120+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":9980,"log_dups_size":0,"ondisk_log_size":9980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40845312,"num_objects":9972,"num_object_clones":0,"num_object_copies":19944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5879,"num_write_kb":23516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5528},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9982","reported_seq":10119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.424868+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:12.424868+0000","last_peered":"2024-09-05T13:16:12.424868+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:12.424868+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":9982,"log_dups_size":0,"ondisk_log_size":9982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6112,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5412},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10051","reported_seq":10189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.324301+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:12.324301+0000","last_peered":"2024-09-05T13:16:12.324301+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:12.324301+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41120227,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6719,"num_write_kb":26876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5475},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650912227,"num_objects":158915,"num_object_clones":0,"num_object_copies":317830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4341,"num_objects_unfound":0,"num_objects_dirty":158915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114714,"num_write_kb":458856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4486,"num_bytes_recovered":18366464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1317683200,"data_stored":1317600101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158996,"ondisk_log_size":158996,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739664,"kb_used_data":460864,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93632176,"statfs":{"total":96636764160,"available":95879348224,"internally_reserved":0,"allocated":471924736,"data_stored":471669251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":285470381},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":898488,"kb_used_data":547560,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93473352,"statfs":{"total":96636764160,"available":95716712448,"internally_reserved":0,"allocated":560701440,"data_stored":560441878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,5,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508404,"kb_used_data":278172,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93863436,"statfs":{"total":96636764160,"available":96116158464,"internally_reserved":0,"allocated":284848128,"data_stored":284516306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235728559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":560689152,"data_stored":560685539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285040640,"data_stored":284964767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471953408,"data_stored":471949795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:16.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:16.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9915", 'reported_seq': 9989, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.223049+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:12.223049+0000', 'last_peered': '2024-09-05T13:16:12.223049+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:12.223049+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 9915, 'log_dups_size': 0, 'ondisk_log_size': 9915, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40611840, 'num_objects': 9915, 'num_object_clones': 0, 'num_object_copies': 19830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9915, 'num_write_kb': 39660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5494}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:16.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:16.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10055", 'reported_seq': 10182, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.763633+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:12.763633+0000', 'last_peered': '2024-09-05T13:16:12.763633+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:12.763633+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41168896, 'num_objects': 10051, 'num_object_clones': 0, 'num_object_copies': 20102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7866, 'num_write_kb': 31464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 5559}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:16.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:16.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10034", 'reported_seq': 10172, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.400249+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:12.400249+0000', 'last_peered': '2024-09-05T13:16:12.400249+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:12.400249+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41078784, 'num_objects': 10029, 'num_object_clones': 0, 'num_object_copies': 20058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6489, 'num_write_kb': 25956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5578}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9782", 'reported_seq': 10110, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.379216+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:12.379216+0000', 'last_peered': '2024-09-05T13:16:12.379216+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:12.379216+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 9782, 'log_dups_size': 0, 'ondisk_log_size': 9782, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40042496, 'num_objects': 9776, 'num_object_clones': 0, 'num_object_copies': 19552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 4341, 'num_objects_unfound': 0, 'num_objects_dirty': 9776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6101, 'num_write_kb': 24404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 94, 'num_bytes_recovered': 385024, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5434}, {'shards': '0,2', 'objects': 4341}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9968", 'reported_seq': 10102, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.806785+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:12.806785+0000', 'last_peered': '2024-09-05T13:16:12.806785+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:12.806785+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 9968, 'log_dups_size': 0, 'ondisk_log_size': 9968, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40800256, 'num_objects': 9961, 'num_object_clones': 0, 'num_object_copies': 19922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7622, 'num_write_kb': 30488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 5510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9908", 'reported_seq': 10045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.410476+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:12.410476+0000', 'last_peered': '2024-09-05T13:16:12.410476+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:12.410476+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 9908, 'log_dups_size': 0, 'ondisk_log_size': 9908, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40550400, 'num_objects': 9900, 'num_object_clones': 0, 'num_object_copies': 19800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6275, 'num_write_kb': 25100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5446}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'9770", 'reported_seq': 9892, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.810355+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:12.810355+0000', 'last_peered': '2024-09-05T13:16:12.810355+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:12.810355+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 9770, 'log_dups_size': 0, 'ondisk_log_size': 9770, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39993344, 'num_objects': 9764, 'num_object_clones': 0, 'num_object_copies': 19528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7279, 'num_write_kb': 29116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 5377}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:16.586 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9879", 'reported_seq': 10019, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.407377+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:12.407377+0000', 'last_peered': '2024-09-05T13:16:12.407377+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:12.407377+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 9879, 'log_dups_size': 0, 'ondisk_log_size': 9879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40456192, 'num_objects': 9877, 'num_object_clones': 0, 'num_object_copies': 19754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9877, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6030, 'num_write_kb': 24120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5458}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:16.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:16.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9980", 'reported_seq': 10167, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.334120+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:12.334120+0000', 'last_peered': '2024-09-05T13:16:12.334120+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:12.334120+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 9980, 'log_dups_size': 0, 'ondisk_log_size': 9980, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40845312, 'num_objects': 9972, 'num_object_clones': 0, 'num_object_copies': 19944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5879, 'num_write_kb': 23516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5528}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:16.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:16.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9982", 'reported_seq': 10119, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.424868+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:12.424868+0000', 'last_peered': '2024-09-05T13:16:12.424868+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:12.424868+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 9982, 'log_dups_size': 0, 'ondisk_log_size': 9982, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40869888, 'num_objects': 9978, 'num_object_clones': 0, 'num_object_copies': 19956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6112, 'num_write_kb': 24448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5412}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:16.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:16.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10051", 'reported_seq': 10189, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:12.324301+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:12.324301+0000', 'last_peered': '2024-09-05T13:16:12.324301+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:12.324301+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41120227, 'num_objects': 10040, 'num_object_clones': 0, 'num_object_copies': 20080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10040, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6719, 'num_write_kb': 26876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5475}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:16.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:17.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 224 20 160916 160896 2.80507 1.53906 0.0549061 0.0278429 2024-09-05T13:16:17.306 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:17.306 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/317834 objects degraded (14.054%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":304,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158917,"data_bytes":650912246,"bytes_used":2198073344,"bytes_avail":287712219136,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":317834,"degraded_ratio":0.14054191810819486,"misplaced_objects":4341,"misplaced_total":317834,"misplaced_ratio":0.013658073082174971,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":134100,"recovering_keys_per_sec":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"write_bytes_sec":1478177,"read_op_per_sec":0,"write_op_per_sec":360},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:17.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:17.334 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:17.882 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:17.885 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:17.908 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-05T13:16:17.594823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656359926,"num_objects":160247,"num_object_clones":0,"num_object_copies":320494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":160247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116046,"num_write_kb":464178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4566,"num_bytes_recovered":18690067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159829,"ondisk_log_size":159829,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2151452,"kb_used_data":1299820,"kb_used_omap":12,"kb_used_meta":851571,"kb_avail":280964068,"statfs":{"total":289910292480,"available":287707205632,"internally_reserved":0,"allocated":1331015680,"data_stored":1330168811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":872008710},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,2,3,3,10,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13897728,"num_objects":3393,"num_object_clones":0,"num_object_copies":6786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4437,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":3393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3393,"num_write_kb":13572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.399888"},"pg_stats":[{"pgid":"2.f","version":"45'10042","reported_seq":10116,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.437070+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:17.437070+0000","last_peered":"2024-09-05T13:16:17.437070+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:17.437070+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132032,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":40168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5621},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9828","reported_seq":9932,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.770401+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:12.770401+0000","last_peered":"2024-09-05T13:16:12.770401+0000","last_clean":"2024-09-05T13:16:12.770401+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:12.770401+0000","last_undegraded":"2024-09-05T13:16:12.770401+0000","last_fullsized":"2024-09-05T13:16:12.770401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9828,"log_dups_size":0,"ondisk_log_size":9828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40230912,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7838,"num_write_kb":31352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10055","reported_seq":10182,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.763633+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:12.763633+0000","last_peered":"2024-09-05T13:16:12.763633+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:12.763633+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7866,"num_write_kb":31464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9963","reported_seq":10032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.744415+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:12.744415+0000","last_peered":"2024-09-05T13:16:12.744415+0000","last_clean":"2024-09-05T13:16:12.744415+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:12.744415+0000","last_undegraded":"2024-09-05T13:16:12.744415+0000","last_fullsized":"2024-09-05T13:16:12.744415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":9963,"log_dups_size":0,"ondisk_log_size":9963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40808448,"num_objects":9963,"num_object_clones":0,"num_object_copies":19926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9963,"num_write_kb":39852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10014","reported_seq":10148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.491490+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:17.491490+0000","last_peered":"2024-09-05T13:16:17.491490+0000","last_clean":"2024-09-05T13:16:17.491490+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:17.491490+0000","last_undegraded":"2024-09-05T13:16:17.491490+0000","last_fullsized":"2024-09-05T13:16:17.491490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6554,"num_write_kb":26216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10179","reported_seq":10316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.465165+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:17.465165+0000","last_peered":"2024-09-05T13:16:17.465165+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:17.465165+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5723},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9910","reported_seq":10396,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.486842+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:17.486842+0000","last_peered":"2024-09-05T13:16:17.486842+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:17.486842+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40566784,"num_objects":9904,"num_object_clones":0,"num_object_copies":19808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":9904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5641},{"shards":"0,2","objects":4262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10156","reported_seq":10287,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.326347+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:17.326347+0000","last_peered":"2024-09-05T13:16:17.326347+0000","last_clean":"2024-09-05T13:16:17.326347+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:17.326347+0000","last_undegraded":"2024-09-05T13:16:17.326347+0000","last_fullsized":"2024-09-05T13:16:17.326347+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41590784,"num_objects":10154,"num_object_clones":0,"num_object_copies":20308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6766,"num_write_kb":27064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9968","reported_seq":10102,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.806785+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:12.806785+0000","last_peered":"2024-09-05T13:16:12.806785+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:12.806785+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800256,"num_objects":9961,"num_object_clones":0,"num_object_copies":19922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7622,"num_write_kb":30488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10040","reported_seq":10178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.403681+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:17.403681+0000","last_peered":"2024-09-05T13:16:17.403681+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:17.403681+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41091072,"num_objects":10032,"num_object_clones":0,"num_object_copies":20064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6407,"num_write_kb":25628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5578},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9962","reported_seq":18861,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.825882+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:12.825882+0000","last_peered":"2024-09-05T13:16:12.825882+0000","last_clean":"2024-09-05T13:16:12.825882+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:12.825882+0000","last_undegraded":"2024-09-05T13:16:12.825882+0000","last_fullsized":"2024-09-05T13:16:12.825882+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":9962,"log_dups_size":0,"ondisk_log_size":9962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40783872,"num_objects":9957,"num_object_clones":0,"num_object_copies":19914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7557,"num_write_kb":30228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9770","reported_seq":9892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:12.810355+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:12.810355+0000","last_peered":"2024-09-05T13:16:12.810355+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:12.810355+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9770,"log_dups_size":0,"ondisk_log_size":9770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39993344,"num_objects":9764,"num_object_clones":0,"num_object_copies":19528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7279,"num_write_kb":29116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5377}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10001","reported_seq":10141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.485389+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:17.485389+0000","last_peered":"2024-09-05T13:16:17.485389+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:17.485389+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40955904,"num_objects":9999,"num_object_clones":0,"num_object_copies":19998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10133","reported_seq":10320,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.471721+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:17.471721+0000","last_peered":"2024-09-05T13:16:17.471721+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:17.471721+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41472000,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5681},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10130","reported_seq":10268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.480864+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:17.480864+0000","last_peered":"2024-09-05T13:16:17.480864+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:17.480864+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5560},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10175","reported_seq":10314,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.522336+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:17.522336+0000","last_peered":"2024-09-05T13:16:17.522336+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:17.522336+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41628131,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5599},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":656359907,"num_objects":160245,"num_object_clones":0,"num_object_copies":320490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":160245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116044,"num_write_kb":464176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4565,"num_bytes_recovered":18690048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1331474432,"data_stored":1331391333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159826,"ondisk_log_size":159826,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739664,"kb_used_data":460864,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93632176,"statfs":{"total":96636764160,"available":95879348224,"internally_reserved":0,"allocated":471924736,"data_stored":471669251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":285470381},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":892288,"kb_used_data":553840,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93479552,"statfs":{"total":96636764160,"available":95723061248,"internally_reserved":0,"allocated":567132160,"data_stored":566872598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,1,2,1,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519500,"kb_used_data":285116,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93852340,"statfs":{"total":96636764160,"available":96104796160,"internally_reserved":0,"allocated":291958784,"data_stored":291626962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239988399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":567279616,"data_stored":567276003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":292241408,"data_stored":292165535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471953408,"data_stored":471949795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:18.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 225 20 161335 161315 2.79988 1.63672 0.0679509 0.027896 2024-09-05T13:16:19.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 226 20 161795 161775 2.79544 1.79688 0.0745968 0.0279404 2024-09-05T13:16:20.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 227 20 162239 162219 2.79077 1.73438 0.0967971 0.0279854 2024-09-05T13:16:20.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:21.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 228 20 162658 162638 2.7857 1.63672 0.0174662 0.0280351 2024-09-05T13:16:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:21.463 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:21.488 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-05T13:16:19.595699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659755510,"num_objects":161076,"num_object_clones":0,"num_object_copies":322152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":161076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116875,"num_write_kb":467494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4566,"num_bytes_recovered":18690067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160458,"ondisk_log_size":160458,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2143236,"kb_used_data":1304084,"kb_used_omap":12,"kb_used_meta":839091,"kb_avail":280972284,"statfs":{"total":289910292480,"available":287715618816,"internally_reserved":0,"allocated":1335382016,"data_stored":1334535147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":859229190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,2,3,2,11,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17293312,"num_objects":4222,"num_object_clones":0,"num_object_copies":8444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4437,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":4222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4222,"num_write_kb":16888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.399169"},"pg_stats":[{"pgid":"2.f","version":"45'10042","reported_seq":10116,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.437070+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:17.437070+0000","last_peered":"2024-09-05T13:16:17.437070+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:17.437070+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132032,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":40168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5621},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9979","reported_seq":10083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.848930+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:17.848930+0000","last_peered":"2024-09-05T13:16:17.848930+0000","last_clean":"2024-09-05T13:16:17.848930+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:17.848930+0000","last_undegraded":"2024-09-05T13:16:17.848930+0000","last_fullsized":"2024-09-05T13:16:17.848930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9979,"log_dups_size":0,"ondisk_log_size":9979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40849408,"num_objects":9973,"num_object_clones":0,"num_object_copies":19946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7989,"num_write_kb":31956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10194","reported_seq":10321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.716182+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:17.716182+0000","last_peered":"2024-09-05T13:16:17.716182+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:17.716182+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41738240,"num_objects":10190,"num_object_clones":0,"num_object_copies":20380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8005,"num_write_kb":32020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5698}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10086","reported_seq":10155,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.695893+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:17.695893+0000","last_peered":"2024-09-05T13:16:17.695893+0000","last_clean":"2024-09-05T13:16:17.695893+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:17.695893+0000","last_undegraded":"2024-09-05T13:16:17.695893+0000","last_fullsized":"2024-09-05T13:16:17.695893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41312256,"num_objects":10086,"num_object_clones":0,"num_object_copies":20172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10086,"num_write_kb":40344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10014","reported_seq":10148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.491490+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:17.491490+0000","last_peered":"2024-09-05T13:16:17.491490+0000","last_clean":"2024-09-05T13:16:17.491490+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:17.491490+0000","last_undegraded":"2024-09-05T13:16:17.491490+0000","last_fullsized":"2024-09-05T13:16:17.491490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6554,"num_write_kb":26216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10179","reported_seq":10316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.465165+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:17.465165+0000","last_peered":"2024-09-05T13:16:17.465165+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:17.465165+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5723},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9910","reported_seq":10396,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.486842+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:17.486842+0000","last_peered":"2024-09-05T13:16:17.486842+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:17.486842+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40566784,"num_objects":9904,"num_object_clones":0,"num_object_copies":19808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":9904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5641},{"shards":"0,2","objects":4262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10156","reported_seq":10287,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.326347+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:17.326347+0000","last_peered":"2024-09-05T13:16:17.326347+0000","last_clean":"2024-09-05T13:16:17.326347+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:17.326347+0000","last_undegraded":"2024-09-05T13:16:17.326347+0000","last_fullsized":"2024-09-05T13:16:17.326347+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41590784,"num_objects":10154,"num_object_clones":0,"num_object_copies":20308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6766,"num_write_kb":27064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10114","reported_seq":10248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.826335+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:17.826335+0000","last_peered":"2024-09-05T13:16:17.826335+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:17.826335+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41398272,"num_objects":10107,"num_object_clones":0,"num_object_copies":20214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":31072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5656}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10040","reported_seq":10178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.403681+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:17.403681+0000","last_peered":"2024-09-05T13:16:17.403681+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:17.403681+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41091072,"num_objects":10032,"num_object_clones":0,"num_object_copies":20064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6407,"num_write_kb":25628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5578},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10092","reported_seq":18991,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.879636+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:17.879636+0000","last_peered":"2024-09-05T13:16:17.879636+0000","last_clean":"2024-09-05T13:16:17.879636+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:17.879636+0000","last_undegraded":"2024-09-05T13:16:17.879636+0000","last_fullsized":"2024-09-05T13:16:17.879636+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41316352,"num_objects":10087,"num_object_clones":0,"num_object_copies":20174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7687,"num_write_kb":30748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9910","reported_seq":10032,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.868191+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:17.868191+0000","last_peered":"2024-09-05T13:16:17.868191+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:17.868191+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40566784,"num_objects":9904,"num_object_clones":0,"num_object_copies":19808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7419,"num_write_kb":29676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10001","reported_seq":10141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.485389+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:17.485389+0000","last_peered":"2024-09-05T13:16:17.485389+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:17.485389+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40955904,"num_objects":9999,"num_object_clones":0,"num_object_copies":19998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10133","reported_seq":10320,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.471721+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:17.471721+0000","last_peered":"2024-09-05T13:16:17.471721+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:17.471721+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41472000,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5681},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10130","reported_seq":10268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.480864+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:17.480864+0000","last_peered":"2024-09-05T13:16:17.480864+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:17.480864+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5560},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10175","reported_seq":10314,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.522336+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:17.522336+0000","last_peered":"2024-09-05T13:16:17.522336+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:17.522336+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41628131,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5599},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659755491,"num_objects":161074,"num_object_clones":0,"num_object_copies":322148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":161074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116873,"num_write_kb":467492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4565,"num_bytes_recovered":18690048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1335808000,"data_stored":1335724901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160455,"ondisk_log_size":160455,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731448,"kb_used_data":465128,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93640392,"statfs":{"total":96636764160,"available":95887761408,"internally_reserved":0,"allocated":476291072,"data_stored":476035587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":892288,"kb_used_data":553840,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93479552,"statfs":{"total":96636764160,"available":95723061248,"internally_reserved":0,"allocated":567132160,"data_stored":566872598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,1,2,1,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519500,"kb_used_data":285116,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93852340,"statfs":{"total":96636764160,"available":96104796160,"internally_reserved":0,"allocated":291958784,"data_stored":291626962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239988399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":567279616,"data_stored":567276003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":292241408,"data_stored":292165535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":476286976,"data_stored":476283363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:21.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:22.033 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:22.056 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-05T13:16:21.596460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659755510,"num_objects":161076,"num_object_clones":0,"num_object_copies":322152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":161076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116875,"num_write_kb":467494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4566,"num_bytes_recovered":18690067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160458,"ondisk_log_size":160458,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2153992,"kb_used_data":1310680,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":280961528,"statfs":{"total":289910292480,"available":287704604672,"internally_reserved":0,"allocated":1342136320,"data_stored":1341289451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":863489030},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,2,2,4,6,11,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17293312,"num_objects":4222,"num_object_clones":0,"num_object_copies":8444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4437,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":4222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4222,"num_write_kb":16888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004219"},"pg_stats":[{"pgid":"2.f","version":"45'10042","reported_seq":10116,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.437070+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:17.437070+0000","last_peered":"2024-09-05T13:16:17.437070+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:17.437070+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132032,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":40168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5621},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9979","reported_seq":10083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.848930+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:17.848930+0000","last_peered":"2024-09-05T13:16:17.848930+0000","last_clean":"2024-09-05T13:16:17.848930+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:17.848930+0000","last_undegraded":"2024-09-05T13:16:17.848930+0000","last_fullsized":"2024-09-05T13:16:17.848930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9979,"log_dups_size":0,"ondisk_log_size":9979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40849408,"num_objects":9973,"num_object_clones":0,"num_object_copies":19946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7989,"num_write_kb":31956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10194","reported_seq":10321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.716182+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:17.716182+0000","last_peered":"2024-09-05T13:16:17.716182+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:17.716182+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41738240,"num_objects":10190,"num_object_clones":0,"num_object_copies":20380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8005,"num_write_kb":32020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5698}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10086","reported_seq":10155,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.695893+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:17.695893+0000","last_peered":"2024-09-05T13:16:17.695893+0000","last_clean":"2024-09-05T13:16:17.695893+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:17.695893+0000","last_undegraded":"2024-09-05T13:16:17.695893+0000","last_fullsized":"2024-09-05T13:16:17.695893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41312256,"num_objects":10086,"num_object_clones":0,"num_object_copies":20172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10086,"num_write_kb":40344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10014","reported_seq":10148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.491490+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:17.491490+0000","last_peered":"2024-09-05T13:16:17.491490+0000","last_clean":"2024-09-05T13:16:17.491490+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:17.491490+0000","last_undegraded":"2024-09-05T13:16:17.491490+0000","last_fullsized":"2024-09-05T13:16:17.491490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6554,"num_write_kb":26216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10179","reported_seq":10316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.465165+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:17.465165+0000","last_peered":"2024-09-05T13:16:17.465165+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:17.465165+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5723},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9910","reported_seq":10396,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.486842+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:17.486842+0000","last_peered":"2024-09-05T13:16:17.486842+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:17.486842+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40566784,"num_objects":9904,"num_object_clones":0,"num_object_copies":19808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":9904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5641},{"shards":"0,2","objects":4262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10156","reported_seq":10287,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.326347+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:17.326347+0000","last_peered":"2024-09-05T13:16:17.326347+0000","last_clean":"2024-09-05T13:16:17.326347+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:17.326347+0000","last_undegraded":"2024-09-05T13:16:17.326347+0000","last_fullsized":"2024-09-05T13:16:17.326347+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41590784,"num_objects":10154,"num_object_clones":0,"num_object_copies":20308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6766,"num_write_kb":27064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10114","reported_seq":10248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.826335+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:17.826335+0000","last_peered":"2024-09-05T13:16:17.826335+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:17.826335+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41398272,"num_objects":10107,"num_object_clones":0,"num_object_copies":20214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":31072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5656}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10040","reported_seq":10178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.403681+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:17.403681+0000","last_peered":"2024-09-05T13:16:17.403681+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:17.403681+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41091072,"num_objects":10032,"num_object_clones":0,"num_object_copies":20064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6407,"num_write_kb":25628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5578},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10092","reported_seq":18991,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.879636+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:17.879636+0000","last_peered":"2024-09-05T13:16:17.879636+0000","last_clean":"2024-09-05T13:16:17.879636+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:17.879636+0000","last_undegraded":"2024-09-05T13:16:17.879636+0000","last_fullsized":"2024-09-05T13:16:17.879636+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41316352,"num_objects":10087,"num_object_clones":0,"num_object_copies":20174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7687,"num_write_kb":30748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9910","reported_seq":10032,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.868191+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:17.868191+0000","last_peered":"2024-09-05T13:16:17.868191+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:17.868191+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40566784,"num_objects":9904,"num_object_clones":0,"num_object_copies":19808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7419,"num_write_kb":29676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10001","reported_seq":10141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.485389+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:17.485389+0000","last_peered":"2024-09-05T13:16:17.485389+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:17.485389+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40955904,"num_objects":9999,"num_object_clones":0,"num_object_copies":19998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10133","reported_seq":10320,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.471721+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:17.471721+0000","last_peered":"2024-09-05T13:16:17.471721+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:17.471721+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41472000,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5681},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10130","reported_seq":10268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.480864+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:17.480864+0000","last_peered":"2024-09-05T13:16:17.480864+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:17.480864+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5560},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10175","reported_seq":10314,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.522336+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:17.522336+0000","last_peered":"2024-09-05T13:16:17.522336+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:17.522336+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41628131,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5599},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659755491,"num_objects":161074,"num_object_clones":0,"num_object_copies":322148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":161074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116873,"num_write_kb":467492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4565,"num_bytes_recovered":18690048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1342713856,"data_stored":1342630757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160455,"ondisk_log_size":160455,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731448,"kb_used_data":465128,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93640392,"statfs":{"total":96636764160,"available":95887761408,"internally_reserved":0,"allocated":476291072,"data_stored":476035587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":892288,"kb_used_data":553840,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93479552,"statfs":{"total":96636764160,"available":95723061248,"internally_reserved":0,"allocated":567132160,"data_stored":566872598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,1,2,1,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530256,"kb_used_data":291712,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93841584,"statfs":{"total":96636764160,"available":96093782016,"internally_reserved":0,"allocated":298713088,"data_stored":298381266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,4,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":567279616,"data_stored":567276003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299147264,"data_stored":299071391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":476286976,"data_stored":476283363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:22.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10042", 'reported_seq': 10116, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.437070+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:17.437070+0000', 'last_peered': '2024-09-05T13:16:17.437070+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:17.437070+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41132032, 'num_objects': 10042, 'num_object_clones': 0, 'num_object_copies': 20084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10042, 'num_write_kb': 40168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5621}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10194", 'reported_seq': 10321, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.716182+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:17.716182+0000', 'last_peered': '2024-09-05T13:16:17.716182+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:17.716182+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41738240, 'num_objects': 10190, 'num_object_clones': 0, 'num_object_copies': 20380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8005, 'num_write_kb': 32020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 5698}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10179", 'reported_seq': 10316, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.465165+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:17.465165+0000', 'last_peered': '2024-09-05T13:16:17.465165+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:17.465165+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41672704, 'num_objects': 10174, 'num_object_clones': 0, 'num_object_copies': 20348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10174, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6634, 'num_write_kb': 26536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5723}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9910", 'reported_seq': 10396, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.486842+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:17.486842+0000', 'last_peered': '2024-09-05T13:16:17.486842+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:17.486842+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 9910, 'log_dups_size': 0, 'ondisk_log_size': 9910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40566784, 'num_objects': 9904, 'num_object_clones': 0, 'num_object_copies': 19808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 4262, 'num_objects_unfound': 0, 'num_objects_dirty': 9904, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6229, 'num_write_kb': 24916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 173, 'num_bytes_recovered': 708608, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5641}, {'shards': '0,2', 'objects': 4262}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:22.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10114", 'reported_seq': 10248, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.826335+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:17.826335+0000', 'last_peered': '2024-09-05T13:16:17.826335+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:17.826335+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41398272, 'num_objects': 10107, 'num_object_clones': 0, 'num_object_copies': 20214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7768, 'num_write_kb': 31072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 5656}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10040", 'reported_seq': 10178, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.403681+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:17.403681+0000', 'last_peered': '2024-09-05T13:16:17.403681+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:17.403681+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41091072, 'num_objects': 10032, 'num_object_clones': 0, 'num_object_copies': 20064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6407, 'num_write_kb': 25628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5578}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'9910", 'reported_seq': 10032, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.868191+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:17.868191+0000', 'last_peered': '2024-09-05T13:16:17.868191+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:17.868191+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 9910, 'log_dups_size': 0, 'ondisk_log_size': 9910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40566784, 'num_objects': 9904, 'num_object_clones': 0, 'num_object_copies': 19808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9904, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7419, 'num_write_kb': 29676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 5517}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10001", 'reported_seq': 10141, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.485389+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:17.485389+0000', 'last_peered': '2024-09-05T13:16:17.485389+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:17.485389+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40955904, 'num_objects': 9999, 'num_object_clones': 0, 'num_object_copies': 19998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9999, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6152, 'num_write_kb': 24608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5580}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10133", 'reported_seq': 10320, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.471721+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:17.471721+0000', 'last_peered': '2024-09-05T13:16:17.471721+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:17.471721+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41472000, 'num_objects': 10125, 'num_object_clones': 0, 'num_object_copies': 20250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10125, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6032, 'num_write_kb': 24128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5681}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:22.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10130", 'reported_seq': 10268, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.480864+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:17.480864+0000', 'last_peered': '2024-09-05T13:16:17.480864+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:17.480864+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41476096, 'num_objects': 10126, 'num_object_clones': 0, 'num_object_copies': 20252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6260, 'num_write_kb': 25040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5560}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:22.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:22.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10175", 'reported_seq': 10314, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:17.522336+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:17.522336+0000', 'last_peered': '2024-09-05T13:16:17.522336+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:17.522336+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41628131, 'num_objects': 10164, 'num_object_clones': 0, 'num_object_copies': 20328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6843, 'num_write_kb': 27372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5599}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:22.060 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:22.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 229 20 163053 163033 2.78028 1.54297 0.0143292 0.0280919 2024-09-05T13:16:22.762 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:22.762 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/322152 objects degraded (13.866%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":309,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":161076,"data_bytes":659755510,"bytes_used":2205687808,"bytes_avail":287704604672,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":322152,"degraded_ratio":0.13865814894832254,"misplaced_objects":4262,"misplaced_total":322152,"misplaced_ratio":0.013229779731306962,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59029,"recovering_keys_per_sec":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"write_bytes_sec":1440602,"read_op_per_sec":0,"write_op_per_sec":351},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:22.790 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:22.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:23.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 230 20 163444 163424 2.77483 1.52734 0.0120333 0.028145 2024-09-05T13:16:23.361 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:23.361 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:23.385 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-05T13:16:21.596460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659755510,"num_objects":161076,"num_object_clones":0,"num_object_copies":322152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":161076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116875,"num_write_kb":467494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4566,"num_bytes_recovered":18690067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160458,"ondisk_log_size":160458,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2153992,"kb_used_data":1310680,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":280961528,"statfs":{"total":289910292480,"available":287704604672,"internally_reserved":0,"allocated":1342136320,"data_stored":1341289451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":863489030},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,2,2,4,6,11,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17293312,"num_objects":4222,"num_object_clones":0,"num_object_copies":8444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4437,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":4222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4222,"num_write_kb":16888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004219"},"pg_stats":[{"pgid":"2.f","version":"45'10042","reported_seq":10116,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.437070+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:17.437070+0000","last_peered":"2024-09-05T13:16:17.437070+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:17.437070+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41132032,"num_objects":10042,"num_object_clones":0,"num_object_copies":20084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":40168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5621},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9979","reported_seq":10083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.848930+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:17.848930+0000","last_peered":"2024-09-05T13:16:17.848930+0000","last_clean":"2024-09-05T13:16:17.848930+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:17.848930+0000","last_undegraded":"2024-09-05T13:16:17.848930+0000","last_fullsized":"2024-09-05T13:16:17.848930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":9979,"log_dups_size":0,"ondisk_log_size":9979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40849408,"num_objects":9973,"num_object_clones":0,"num_object_copies":19946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7989,"num_write_kb":31956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10194","reported_seq":10321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.716182+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:17.716182+0000","last_peered":"2024-09-05T13:16:17.716182+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:17.716182+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41738240,"num_objects":10190,"num_object_clones":0,"num_object_copies":20380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8005,"num_write_kb":32020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5698}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10086","reported_seq":10155,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.695893+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:17.695893+0000","last_peered":"2024-09-05T13:16:17.695893+0000","last_clean":"2024-09-05T13:16:17.695893+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:17.695893+0000","last_undegraded":"2024-09-05T13:16:17.695893+0000","last_fullsized":"2024-09-05T13:16:17.695893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41312256,"num_objects":10086,"num_object_clones":0,"num_object_copies":20172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10086,"num_write_kb":40344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10014","reported_seq":10148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.491490+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:17.491490+0000","last_peered":"2024-09-05T13:16:17.491490+0000","last_clean":"2024-09-05T13:16:17.491490+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:17.491490+0000","last_undegraded":"2024-09-05T13:16:17.491490+0000","last_fullsized":"2024-09-05T13:16:17.491490+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6554,"num_write_kb":26216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10179","reported_seq":10316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.465165+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:17.465165+0000","last_peered":"2024-09-05T13:16:17.465165+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:17.465165+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5723},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9910","reported_seq":10396,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.486842+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:17.486842+0000","last_peered":"2024-09-05T13:16:17.486842+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:17.486842+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40566784,"num_objects":9904,"num_object_clones":0,"num_object_copies":19808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":9904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5641},{"shards":"0,2","objects":4262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10156","reported_seq":10287,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.326347+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:17.326347+0000","last_peered":"2024-09-05T13:16:17.326347+0000","last_clean":"2024-09-05T13:16:17.326347+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:17.326347+0000","last_undegraded":"2024-09-05T13:16:17.326347+0000","last_fullsized":"2024-09-05T13:16:17.326347+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41590784,"num_objects":10154,"num_object_clones":0,"num_object_copies":20308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6766,"num_write_kb":27064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10114","reported_seq":10248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.826335+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:17.826335+0000","last_peered":"2024-09-05T13:16:17.826335+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:17.826335+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41398272,"num_objects":10107,"num_object_clones":0,"num_object_copies":20214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7768,"num_write_kb":31072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5656}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10040","reported_seq":10178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.403681+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:17.403681+0000","last_peered":"2024-09-05T13:16:17.403681+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:17.403681+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41091072,"num_objects":10032,"num_object_clones":0,"num_object_copies":20064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6407,"num_write_kb":25628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5578},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10092","reported_seq":18991,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:17.879636+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:17.879636+0000","last_peered":"2024-09-05T13:16:17.879636+0000","last_clean":"2024-09-05T13:16:17.879636+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:17.879636+0000","last_undegraded":"2024-09-05T13:16:17.879636+0000","last_fullsized":"2024-09-05T13:16:17.879636+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41316352,"num_objects":10087,"num_object_clones":0,"num_object_copies":20174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7687,"num_write_kb":30748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9910","reported_seq":10032,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.868191+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:17.868191+0000","last_peered":"2024-09-05T13:16:17.868191+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:17.868191+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":9910,"log_dups_size":0,"ondisk_log_size":9910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40566784,"num_objects":9904,"num_object_clones":0,"num_object_copies":19808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7419,"num_write_kb":29676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10001","reported_seq":10141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.485389+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:17.485389+0000","last_peered":"2024-09-05T13:16:17.485389+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:17.485389+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40955904,"num_objects":9999,"num_object_clones":0,"num_object_copies":19998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10133","reported_seq":10320,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.471721+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:17.471721+0000","last_peered":"2024-09-05T13:16:17.471721+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:17.471721+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41472000,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5681},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10130","reported_seq":10268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.480864+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:17.480864+0000","last_peered":"2024-09-05T13:16:17.480864+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:17.480864+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41476096,"num_objects":10126,"num_object_clones":0,"num_object_copies":20252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6260,"num_write_kb":25040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5560},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10175","reported_seq":10314,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:17.522336+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:17.522336+0000","last_peered":"2024-09-05T13:16:17.522336+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:17.522336+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41628131,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5599},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659755491,"num_objects":161074,"num_object_clones":0,"num_object_copies":322148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4262,"num_objects_unfound":0,"num_objects_dirty":161074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116873,"num_write_kb":467492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4565,"num_bytes_recovered":18690048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1342713856,"data_stored":1342630757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160455,"ondisk_log_size":160455,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731448,"kb_used_data":465128,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93640392,"statfs":{"total":96636764160,"available":95887761408,"internally_reserved":0,"allocated":476291072,"data_stored":476035587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":892288,"kb_used_data":553840,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93479552,"statfs":{"total":96636764160,"available":95723061248,"internally_reserved":0,"allocated":567132160,"data_stored":566872598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,1,2,1,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530256,"kb_used_data":291712,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93841584,"statfs":{"total":96636764160,"available":96093782016,"internally_reserved":0,"allocated":298713088,"data_stored":298381266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,4,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":567279616,"data_stored":567276003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299147264,"data_stored":299071391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":476286976,"data_stored":476283363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:24.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 231 20 163882 163862 2.77023 1.71094 0.032869 0.0281945 2024-09-05T13:16:25.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 232 20 164333 164313 2.76588 1.76172 0.0197493 0.0282372 2024-09-05T13:16:26.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 233 20 164783 164763 2.76155 1.75781 0.0652472 0.0282831 2024-09-05T13:16:26.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:26.953 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:26.955 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:26.978 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-05T13:16:25.598154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668574198,"num_objects":163229,"num_object_clones":0,"num_object_copies":326458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":163229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119028,"num_write_kb":476106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4638,"num_bytes_recovered":18984979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2168836,"kb_used_data":1321364,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280946684,"statfs":{"total":289910292480,"available":287689404416,"internally_reserved":0,"allocated":1353076736,"data_stored":1352229867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":867748870},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,10,10,6,7,4,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17661952,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-151,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":17248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":618496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004483"},"pg_stats":[{"pgid":"2.f","version":"45'10197","reported_seq":10269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.618191+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:22.618191+0000","last_peered":"2024-09-05T13:16:22.618191+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:22.618191+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41766912,"num_objects":10197,"num_object_clones":0,"num_object_copies":20394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10197,"num_write_kb":40788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5776},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10100","reported_seq":10204,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.844959+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:22.844959+0000","last_peered":"2024-09-05T13:16:22.844959+0000","last_clean":"2024-09-05T13:16:22.844959+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:22.844959+0000","last_undegraded":"2024-09-05T13:16:22.844959+0000","last_fullsized":"2024-09-05T13:16:22.844959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41345024,"num_objects":10094,"num_object_clones":0,"num_object_copies":20188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8110,"num_write_kb":32440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10336","reported_seq":10463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.825524+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:22.825524+0000","last_peered":"2024-09-05T13:16:22.825524+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:22.825524+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":32588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10224","reported_seq":10293,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.814085+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:22.814085+0000","last_peered":"2024-09-05T13:16:22.814085+0000","last_clean":"2024-09-05T13:16:22.814085+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:22.814085+0000","last_undegraded":"2024-09-05T13:16:22.814085+0000","last_fullsized":"2024-09-05T13:16:22.814085+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41877504,"num_objects":10224,"num_object_clones":0,"num_object_copies":20448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10224,"num_write_kb":40896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10149","reported_seq":10283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.598231+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:22.598231+0000","last_peered":"2024-09-05T13:16:22.598231+0000","last_clean":"2024-09-05T13:16:22.598231+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:22.598231+0000","last_undegraded":"2024-09-05T13:16:22.598231+0000","last_fullsized":"2024-09-05T13:16:22.598231+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41541632,"num_objects":10142,"num_object_clones":0,"num_object_copies":20284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6689,"num_write_kb":26756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10297","reported_seq":10435,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.583148+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:22.583148+0000","last_peered":"2024-09-05T13:16:22.583148+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:22.583148+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42156032,"num_objects":10292,"num_object_clones":0,"num_object_copies":20584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10057","reported_seq":10683,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.572729+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:22.572729+0000","last_peered":"2024-09-05T13:16:22.572729+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:22.572729+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6376,"num_write_kb":25504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5860},{"shards":"0,2","objects":4190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10288","reported_seq":10420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.582656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:22.582656+0000","last_peered":"2024-09-05T13:16:22.582656+0000","last_clean":"2024-09-05T13:16:22.582656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:22.582656+0000","last_undegraded":"2024-09-05T13:16:22.582656+0000","last_fullsized":"2024-09-05T13:16:22.582656+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42131456,"num_objects":10286,"num_object_clones":0,"num_object_copies":20572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6898,"num_write_kb":27592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10250","reported_seq":10384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.806719+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:22.806719+0000","last_peered":"2024-09-05T13:16:22.806719+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:22.806719+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41955328,"num_objects":10243,"num_object_clones":0,"num_object_copies":20486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7904,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5792}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10164","reported_seq":10302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.535919+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:22.535919+0000","last_peered":"2024-09-05T13:16:22.535919+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:22.535919+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6531,"num_write_kb":26124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10211","reported_seq":19111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.659782+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:22.659782+0000","last_peered":"2024-09-05T13:16:22.659782+0000","last_clean":"2024-09-05T13:16:22.659782+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:22.659782+0000","last_undegraded":"2024-09-05T13:16:22.659782+0000","last_fullsized":"2024-09-05T13:16:22.659782+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41803776,"num_objects":10206,"num_object_clones":0,"num_object_copies":20412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10028","reported_seq":10151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.859483+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:22.859483+0000","last_peered":"2024-09-05T13:16:22.859483+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:22.859483+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41050112,"num_objects":10022,"num_object_clones":0,"num_object_copies":20044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7537,"num_write_kb":30148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10146","reported_seq":10285,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.559016+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:22.559016+0000","last_peered":"2024-09-05T13:16:22.559016+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:22.559016+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":10144,"num_object_clones":0,"num_object_copies":20288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6297,"num_write_kb":25188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10268","reported_seq":10455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.570711+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:22.570711+0000","last_peered":"2024-09-05T13:16:22.570711+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:22.570711+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42024960,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6167,"num_write_kb":24668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5816},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10276","reported_seq":10411,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.598374+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:22.598374+0000","last_peered":"2024-09-05T13:16:22.598374+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:22.598374+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42074112,"num_objects":10272,"num_object_clones":0,"num_object_copies":20544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6406,"num_write_kb":25624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5706},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10317","reported_seq":10456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.639679+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:22.639679+0000","last_peered":"2024-09-05T13:16:22.639679+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:22.639679+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42209763,"num_objects":10306,"num_object_clones":0,"num_object_copies":20612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668574179,"num_objects":163227,"num_object_clones":0,"num_object_copies":326454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":163227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119026,"num_write_kb":476104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4637,"num_bytes_recovered":18984960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1353715712,"data_stored":1353632613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735640,"kb_used_data":469320,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93636200,"statfs":{"total":96636764160,"available":95883468800,"internally_reserved":0,"allocated":480583680,"data_stored":480328195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902940,"kb_used_data":560332,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93468900,"statfs":{"total":96636764160,"available":95712153600,"internally_reserved":0,"allocated":573779968,"data_stored":573520406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,5,1,6,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530256,"kb_used_data":291712,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93841584,"statfs":{"total":96636764160,"available":96093782016,"internally_reserved":0,"allocated":298713088,"data_stored":298381266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,4,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":573992960,"data_stored":573989347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299147264,"data_stored":299071391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480575488,"data_stored":480571875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:26.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:27.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 234 20 165187 165167 2.7565 1.57812 0.0469307 0.0283342 2024-09-05T13:16:27.530 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:27.530 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:27.553 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-05T13:16:25.598154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668574198,"num_objects":163229,"num_object_clones":0,"num_object_copies":326458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":163229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119028,"num_write_kb":476106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4638,"num_bytes_recovered":18984979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2168836,"kb_used_data":1321364,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280946684,"statfs":{"total":289910292480,"available":287689404416,"internally_reserved":0,"allocated":1353076736,"data_stored":1352229867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":867748870},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,10,10,6,7,4,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17661952,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-151,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":17248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":618496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004483"},"pg_stats":[{"pgid":"2.f","version":"45'10197","reported_seq":10269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.618191+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:22.618191+0000","last_peered":"2024-09-05T13:16:22.618191+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:22.618191+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41766912,"num_objects":10197,"num_object_clones":0,"num_object_copies":20394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10197,"num_write_kb":40788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5776},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10100","reported_seq":10204,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.844959+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:22.844959+0000","last_peered":"2024-09-05T13:16:22.844959+0000","last_clean":"2024-09-05T13:16:22.844959+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:22.844959+0000","last_undegraded":"2024-09-05T13:16:22.844959+0000","last_fullsized":"2024-09-05T13:16:22.844959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41345024,"num_objects":10094,"num_object_clones":0,"num_object_copies":20188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8110,"num_write_kb":32440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10336","reported_seq":10463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.825524+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:22.825524+0000","last_peered":"2024-09-05T13:16:22.825524+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:22.825524+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":32588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10224","reported_seq":10293,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.814085+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:22.814085+0000","last_peered":"2024-09-05T13:16:22.814085+0000","last_clean":"2024-09-05T13:16:22.814085+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:22.814085+0000","last_undegraded":"2024-09-05T13:16:22.814085+0000","last_fullsized":"2024-09-05T13:16:22.814085+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41877504,"num_objects":10224,"num_object_clones":0,"num_object_copies":20448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10224,"num_write_kb":40896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10149","reported_seq":10283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.598231+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:22.598231+0000","last_peered":"2024-09-05T13:16:22.598231+0000","last_clean":"2024-09-05T13:16:22.598231+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:22.598231+0000","last_undegraded":"2024-09-05T13:16:22.598231+0000","last_fullsized":"2024-09-05T13:16:22.598231+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41541632,"num_objects":10142,"num_object_clones":0,"num_object_copies":20284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6689,"num_write_kb":26756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10297","reported_seq":10435,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.583148+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:22.583148+0000","last_peered":"2024-09-05T13:16:22.583148+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:22.583148+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42156032,"num_objects":10292,"num_object_clones":0,"num_object_copies":20584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10057","reported_seq":10683,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.572729+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:22.572729+0000","last_peered":"2024-09-05T13:16:22.572729+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:22.572729+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6376,"num_write_kb":25504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5860},{"shards":"0,2","objects":4190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10288","reported_seq":10420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.582656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:22.582656+0000","last_peered":"2024-09-05T13:16:22.582656+0000","last_clean":"2024-09-05T13:16:22.582656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:22.582656+0000","last_undegraded":"2024-09-05T13:16:22.582656+0000","last_fullsized":"2024-09-05T13:16:22.582656+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42131456,"num_objects":10286,"num_object_clones":0,"num_object_copies":20572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6898,"num_write_kb":27592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10250","reported_seq":10384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.806719+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:22.806719+0000","last_peered":"2024-09-05T13:16:22.806719+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:22.806719+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41955328,"num_objects":10243,"num_object_clones":0,"num_object_copies":20486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7904,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5792}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10164","reported_seq":10302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.535919+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:22.535919+0000","last_peered":"2024-09-05T13:16:22.535919+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:22.535919+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6531,"num_write_kb":26124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10211","reported_seq":19111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.659782+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:22.659782+0000","last_peered":"2024-09-05T13:16:22.659782+0000","last_clean":"2024-09-05T13:16:22.659782+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:22.659782+0000","last_undegraded":"2024-09-05T13:16:22.659782+0000","last_fullsized":"2024-09-05T13:16:22.659782+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41803776,"num_objects":10206,"num_object_clones":0,"num_object_copies":20412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10028","reported_seq":10151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.859483+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:22.859483+0000","last_peered":"2024-09-05T13:16:22.859483+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:22.859483+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41050112,"num_objects":10022,"num_object_clones":0,"num_object_copies":20044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7537,"num_write_kb":30148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10146","reported_seq":10285,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.559016+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:22.559016+0000","last_peered":"2024-09-05T13:16:22.559016+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:22.559016+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":10144,"num_object_clones":0,"num_object_copies":20288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6297,"num_write_kb":25188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10268","reported_seq":10455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.570711+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:22.570711+0000","last_peered":"2024-09-05T13:16:22.570711+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:22.570711+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42024960,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6167,"num_write_kb":24668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5816},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10276","reported_seq":10411,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.598374+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:22.598374+0000","last_peered":"2024-09-05T13:16:22.598374+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:22.598374+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42074112,"num_objects":10272,"num_object_clones":0,"num_object_copies":20544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6406,"num_write_kb":25624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5706},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10317","reported_seq":10456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.639679+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:22.639679+0000","last_peered":"2024-09-05T13:16:22.639679+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:22.639679+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42209763,"num_objects":10306,"num_object_clones":0,"num_object_copies":20612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668574179,"num_objects":163227,"num_object_clones":0,"num_object_copies":326454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":163227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119026,"num_write_kb":476104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4637,"num_bytes_recovered":18984960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1353715712,"data_stored":1353632613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735640,"kb_used_data":469320,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93636200,"statfs":{"total":96636764160,"available":95883468800,"internally_reserved":0,"allocated":480583680,"data_stored":480328195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902940,"kb_used_data":560332,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93468900,"statfs":{"total":96636764160,"available":95712153600,"internally_reserved":0,"allocated":573779968,"data_stored":573520406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,5,1,6,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530256,"kb_used_data":291712,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93841584,"statfs":{"total":96636764160,"available":96093782016,"internally_reserved":0,"allocated":298713088,"data_stored":298381266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,4,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":573992960,"data_stored":573989347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299147264,"data_stored":299071391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480575488,"data_stored":480571875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:27.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:27.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10197", 'reported_seq': 10269, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.618191+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:22.618191+0000', 'last_peered': '2024-09-05T13:16:22.618191+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:22.618191+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41766912, 'num_objects': 10197, 'num_object_clones': 0, 'num_object_copies': 20394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10197, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10197, 'num_write_kb': 40788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5776}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:27.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:27.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10336", 'reported_seq': 10463, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.825524+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:22.825524+0000', 'last_peered': '2024-09-05T13:16:22.825524+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:22.825524+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42319872, 'num_objects': 10332, 'num_object_clones': 0, 'num_object_copies': 20664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8147, 'num_write_kb': 32588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 5840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10297", 'reported_seq': 10435, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.583148+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:22.583148+0000', 'last_peered': '2024-09-05T13:16:22.583148+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:22.583148+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42156032, 'num_objects': 10292, 'num_object_clones': 0, 'num_object_copies': 20584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10292, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6752, 'num_write_kb': 27008, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5841}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10057", 'reported_seq': 10683, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.572729+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:22.572729+0000', 'last_peered': '2024-09-05T13:16:22.572729+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:22.572729+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41168896, 'num_objects': 10051, 'num_object_clones': 0, 'num_object_copies': 20102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 4190, 'num_objects_unfound': 0, 'num_objects_dirty': 10051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6376, 'num_write_kb': 25504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 245, 'num_bytes_recovered': 1003520, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5860}, {'shards': '0,2', 'objects': 4190}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10250", 'reported_seq': 10384, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.806719+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:22.806719+0000', 'last_peered': '2024-09-05T13:16:22.806719+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:22.806719+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41955328, 'num_objects': 10243, 'num_object_clones': 0, 'num_object_copies': 20486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10243, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7904, 'num_write_kb': 31616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 5792}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10164", 'reported_seq': 10302, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.535919+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:22.535919+0000', 'last_peered': '2024-09-05T13:16:22.535919+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:22.535919+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41598976, 'num_objects': 10156, 'num_object_clones': 0, 'num_object_copies': 20312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6531, 'num_write_kb': 26124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5702}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10028", 'reported_seq': 10151, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.859483+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:22.859483+0000', 'last_peered': '2024-09-05T13:16:22.859483+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:22.859483+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41050112, 'num_objects': 10022, 'num_object_clones': 0, 'num_object_copies': 20044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7537, 'num_write_kb': 30148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 5635}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:27.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:27.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10146", 'reported_seq': 10285, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.559016+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:22.559016+0000', 'last_peered': '2024-09-05T13:16:22.559016+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:22.559016+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41549824, 'num_objects': 10144, 'num_object_clones': 0, 'num_object_copies': 20288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10144, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6297, 'num_write_kb': 25188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5725}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:27.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:27.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10268", 'reported_seq': 10455, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.570711+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:22.570711+0000', 'last_peered': '2024-09-05T13:16:22.570711+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:22.570711+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42024960, 'num_objects': 10260, 'num_object_clones': 0, 'num_object_copies': 20520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6167, 'num_write_kb': 24668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5816}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:27.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:27.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10276", 'reported_seq': 10411, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.598374+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:22.598374+0000', 'last_peered': '2024-09-05T13:16:22.598374+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:22.598374+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42074112, 'num_objects': 10272, 'num_object_clones': 0, 'num_object_copies': 20544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6406, 'num_write_kb': 25624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5706}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:27.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:27.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10317", 'reported_seq': 10456, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:22.639679+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:22.639679+0000', 'last_peered': '2024-09-05T13:16:22.639679+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:22.639679+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42209763, 'num_objects': 10306, 'num_object_clones': 0, 'num_object_copies': 20612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6985, 'num_write_kb': 27940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5741}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:27.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:28.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 235 20 165579 165559 2.75127 1.53125 0.0471467 0.0283885 2024-09-05T13:16:28.285 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:28.286 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/326458 objects degraded (13.683%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":315,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":163229,"data_bytes":668574198,"bytes_used":2220888064,"bytes_avail":287689404416,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":326458,"degraded_ratio":0.13682923990222326,"misplaced_objects":4190,"misplaced_total":326458,"misplaced_ratio":0.012834729122888702,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":51522,"recovering_keys_per_sec":0,"num_objects_recovered":151,"num_bytes_recovered":618496,"num_keys_recovered":0,"write_bytes_sec":1471279,"read_op_per_sec":0,"write_op_per_sec":359},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:28.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:28.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:28.846 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-05T13:16:27.598812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668574198,"num_objects":163229,"num_object_clones":0,"num_object_copies":326458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":163229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119028,"num_write_kb":476106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4638,"num_bytes_recovered":18984979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2162948,"kb_used_data":1327956,"kb_used_omap":12,"kb_used_meta":834931,"kb_avail":280952572,"statfs":{"total":289910292480,"available":287695433728,"internally_reserved":0,"allocated":1359826944,"data_stored":1358980075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":854969350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,12,7,9,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17661952,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-151,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4312,"num_write_kb":17248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":618496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004630"},"pg_stats":[{"pgid":"2.f","version":"45'10197","reported_seq":10269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.618191+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:22.618191+0000","last_peered":"2024-09-05T13:16:22.618191+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:22.618191+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41766912,"num_objects":10197,"num_object_clones":0,"num_object_copies":20394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10197,"num_write_kb":40788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5776},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10100","reported_seq":10204,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.844959+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:22.844959+0000","last_peered":"2024-09-05T13:16:22.844959+0000","last_clean":"2024-09-05T13:16:22.844959+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:22.844959+0000","last_undegraded":"2024-09-05T13:16:22.844959+0000","last_fullsized":"2024-09-05T13:16:22.844959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41345024,"num_objects":10094,"num_object_clones":0,"num_object_copies":20188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8110,"num_write_kb":32440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10336","reported_seq":10463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.825524+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:22.825524+0000","last_peered":"2024-09-05T13:16:22.825524+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:22.825524+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":32588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10224","reported_seq":10293,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.814085+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:22.814085+0000","last_peered":"2024-09-05T13:16:22.814085+0000","last_clean":"2024-09-05T13:16:22.814085+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:22.814085+0000","last_undegraded":"2024-09-05T13:16:22.814085+0000","last_fullsized":"2024-09-05T13:16:22.814085+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41877504,"num_objects":10224,"num_object_clones":0,"num_object_copies":20448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10224,"num_write_kb":40896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10149","reported_seq":10283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.598231+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:22.598231+0000","last_peered":"2024-09-05T13:16:22.598231+0000","last_clean":"2024-09-05T13:16:22.598231+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:22.598231+0000","last_undegraded":"2024-09-05T13:16:22.598231+0000","last_fullsized":"2024-09-05T13:16:22.598231+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41541632,"num_objects":10142,"num_object_clones":0,"num_object_copies":20284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6689,"num_write_kb":26756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10297","reported_seq":10435,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.583148+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:22.583148+0000","last_peered":"2024-09-05T13:16:22.583148+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:22.583148+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42156032,"num_objects":10292,"num_object_clones":0,"num_object_copies":20584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10057","reported_seq":10683,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.572729+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:22.572729+0000","last_peered":"2024-09-05T13:16:22.572729+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:22.572729+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6376,"num_write_kb":25504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5860},{"shards":"0,2","objects":4190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10288","reported_seq":10420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.582656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:22.582656+0000","last_peered":"2024-09-05T13:16:22.582656+0000","last_clean":"2024-09-05T13:16:22.582656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:22.582656+0000","last_undegraded":"2024-09-05T13:16:22.582656+0000","last_fullsized":"2024-09-05T13:16:22.582656+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42131456,"num_objects":10286,"num_object_clones":0,"num_object_copies":20572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6898,"num_write_kb":27592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10250","reported_seq":10384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.806719+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:22.806719+0000","last_peered":"2024-09-05T13:16:22.806719+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:22.806719+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41955328,"num_objects":10243,"num_object_clones":0,"num_object_copies":20486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7904,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5792}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198203+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198203+0000","last_peered":"2024-09-05T13:16:09.198203+0000","last_clean":"2024-09-05T13:16:09.198203+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198203+0000","last_undegraded":"2024-09-05T13:16:09.198203+0000","last_fullsized":"2024-09-05T13:16:09.198203+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10164","reported_seq":10302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.535919+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:22.535919+0000","last_peered":"2024-09-05T13:16:22.535919+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:22.535919+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41598976,"num_objects":10156,"num_object_clones":0,"num_object_copies":20312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6531,"num_write_kb":26124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10211","reported_seq":19111,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:22.659782+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:22.659782+0000","last_peered":"2024-09-05T13:16:22.659782+0000","last_clean":"2024-09-05T13:16:22.659782+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:22.659782+0000","last_undegraded":"2024-09-05T13:16:22.659782+0000","last_fullsized":"2024-09-05T13:16:22.659782+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41803776,"num_objects":10206,"num_object_clones":0,"num_object_copies":20412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10028","reported_seq":10151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.859483+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:22.859483+0000","last_peered":"2024-09-05T13:16:22.859483+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:22.859483+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41050112,"num_objects":10022,"num_object_clones":0,"num_object_copies":20044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7537,"num_write_kb":30148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10146","reported_seq":10285,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.559016+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:22.559016+0000","last_peered":"2024-09-05T13:16:22.559016+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:22.559016+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":10144,"num_object_clones":0,"num_object_copies":20288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6297,"num_write_kb":25188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5725},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10268","reported_seq":10455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.570711+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:22.570711+0000","last_peered":"2024-09-05T13:16:22.570711+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:22.570711+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42024960,"num_objects":10260,"num_object_clones":0,"num_object_copies":20520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6167,"num_write_kb":24668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5816},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199254+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199254+0000","last_peered":"2024-09-05T13:16:09.199254+0000","last_clean":"2024-09-05T13:16:09.199254+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199254+0000","last_undegraded":"2024-09-05T13:16:09.199254+0000","last_fullsized":"2024-09-05T13:16:09.199254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10276","reported_seq":10411,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.598374+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:22.598374+0000","last_peered":"2024-09-05T13:16:22.598374+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:22.598374+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42074112,"num_objects":10272,"num_object_clones":0,"num_object_copies":20544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6406,"num_write_kb":25624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5706},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10317","reported_seq":10456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:22.639679+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:22.639679+0000","last_peered":"2024-09-05T13:16:22.639679+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:22.639679+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42209763,"num_objects":10306,"num_object_clones":0,"num_object_copies":20612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":668574179,"num_objects":163227,"num_object_clones":0,"num_object_copies":326454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4190,"num_objects_unfound":0,"num_objects_dirty":163227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119026,"num_write_kb":476104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4637,"num_bytes_recovered":18984960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1360625664,"data_stored":1360542565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735640,"kb_used_data":469320,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93636200,"statfs":{"total":96636764160,"available":95883468800,"internally_reserved":0,"allocated":480583680,"data_stored":480328195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":272690861},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902940,"kb_used_data":560332,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93468900,"statfs":{"total":96636764160,"available":95712153600,"internally_reserved":0,"allocated":573779968,"data_stored":573520406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,5,1,6,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524368,"kb_used_data":298304,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93847472,"statfs":{"total":96636764160,"available":96099811328,"internally_reserved":0,"allocated":305463296,"data_stored":305131474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":231468719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":573992960,"data_stored":573989347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306057216,"data_stored":305981343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480575488,"data_stored":480571875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:29.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 236 20 165989 165969 2.7464 1.60156 0.00986936 0.028438 2024-09-05T13:16:30.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 237 20 166427 166407 2.74203 1.71094 0.0806847 0.0284839 2024-09-05T13:16:31.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 238 20 166877 166857 2.73789 1.75781 0.0355178 0.0285267 2024-09-05T13:16:31.883 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:32.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout: 239 20 167319 167299 2.73366 1.72656 0.0187126 0.0285685 2024-09-05T13:16:32.458 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:32.461 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:32.479 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-05T13:16:31.599960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":677528054,"num_objects":165415,"num_object_clones":0,"num_object_copies":330830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4107,"num_objects_unfound":0,"num_objects_dirty":165415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121214,"num_write_kb":484850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4721,"num_bytes_recovered":19324947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2196784,"kb_used_data":1345148,"kb_used_omap":12,"kb_used_meta":851571,"kb_avail":280918736,"statfs":{"total":289910292480,"available":287660785664,"internally_reserved":0,"allocated":1377431552,"data_stored":1376584683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":872008710},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,5,2,11,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17772544,"num_objects":4339,"num_object_clones":0,"num_object_copies":8678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-155,"num_objects_unfound":0,"num_objects_dirty":4339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":634880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004261"},"pg_stats":[{"pgid":"2.f","version":"45'10332","reported_seq":10406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.768482+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:27.768482+0000","last_peered":"2024-09-05T13:16:27.768482+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:27.768482+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10332,"num_write_kb":41328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10250","reported_seq":10354,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.746101+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:27.746101+0000","last_peered":"2024-09-05T13:16:27.746101+0000","last_clean":"2024-09-05T13:16:27.746101+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:27.746101+0000","last_undegraded":"2024-09-05T13:16:27.746101+0000","last_fullsized":"2024-09-05T13:16:27.746101+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41959424,"num_objects":10244,"num_object_clones":0,"num_object_copies":20488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8260,"num_write_kb":33040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10472","reported_seq":10599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.905956+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:27.905956+0000","last_peered":"2024-09-05T13:16:27.905956+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:27.905956+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42876928,"num_objects":10468,"num_object_clones":0,"num_object_copies":20936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8283,"num_write_kb":33132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5976}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10362","reported_seq":10431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.843157+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:27.843157+0000","last_peered":"2024-09-05T13:16:27.843157+0000","last_clean":"2024-09-05T13:16:27.843157+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:27.843157+0000","last_undegraded":"2024-09-05T13:16:27.843157+0000","last_fullsized":"2024-09-05T13:16:27.843157+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42442752,"num_objects":10362,"num_object_clones":0,"num_object_copies":20724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10362,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10302","reported_seq":10436,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.693663+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:27.693663+0000","last_peered":"2024-09-05T13:16:27.693663+0000","last_clean":"2024-09-05T13:16:27.693663+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:27.693663+0000","last_undegraded":"2024-09-05T13:16:27.693663+0000","last_fullsized":"2024-09-05T13:16:27.693663+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42168320,"num_objects":10295,"num_object_clones":0,"num_object_copies":20590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10432","reported_seq":10567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.811097+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:27.811097+0000","last_peered":"2024-09-05T13:16:27.811097+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:27.811097+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42708992,"num_objects":10427,"num_object_clones":0,"num_object_copies":20854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6887,"num_write_kb":27548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5976},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10180","reported_seq":10976,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.744446+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:27.744446+0000","last_peered":"2024-09-05T13:16:27.744446+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:27.744446+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4107,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6499,"num_write_kb":25996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1343488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6066},{"shards":"0,2","objects":4107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10412","reported_seq":10544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.685056+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:27.685056+0000","last_peered":"2024-09-05T13:16:27.685056+0000","last_clean":"2024-09-05T13:16:27.685056+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:27.685056+0000","last_undegraded":"2024-09-05T13:16:27.685056+0000","last_fullsized":"2024-09-05T13:16:27.685056+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42639360,"num_objects":10410,"num_object_clones":0,"num_object_copies":20820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":28088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10392","reported_seq":10526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.883345+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:27.883345+0000","last_peered":"2024-09-05T13:16:27.883345+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:27.883345+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42536960,"num_objects":10385,"num_object_clones":0,"num_object_copies":20770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":32184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5934}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10316","reported_seq":10454,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.809612+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:27.809612+0000","last_peered":"2024-09-05T13:16:27.809612+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:27.809612+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42221568,"num_objects":10308,"num_object_clones":0,"num_object_copies":20616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":26732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5854},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10320","reported_seq":19220,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.723992+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:27.723992+0000","last_peered":"2024-09-05T13:16:27.723992+0000","last_clean":"2024-09-05T13:16:27.723992+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:27.723992+0000","last_undegraded":"2024-09-05T13:16:27.723992+0000","last_fullsized":"2024-09-05T13:16:27.723992+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42250240,"num_objects":10315,"num_object_clones":0,"num_object_copies":20630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7915,"num_write_kb":31660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10164","reported_seq":10287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.839738+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:27.839738+0000","last_peered":"2024-09-05T13:16:27.839738+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:27.839738+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41607168,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7673,"num_write_kb":30692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5771}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10297","reported_seq":10435,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.748467+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:27.748467+0000","last_peered":"2024-09-05T13:16:27.748467+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:27.748467+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42168320,"num_objects":10295,"num_object_clones":0,"num_object_copies":20590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":25792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5876},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10396","reported_seq":10582,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.781258+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:27.781258+0000","last_peered":"2024-09-05T13:16:27.781258+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:27.781258+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42549248,"num_objects":10388,"num_object_clones":0,"num_object_copies":20776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":25180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5944},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10404","reported_seq":10543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.676982+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:27.676982+0000","last_peered":"2024-09-05T13:16:27.676982+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:27.676982+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42598400,"num_objects":10400,"num_object_clones":0,"num_object_copies":20800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6534,"num_write_kb":26136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5834},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10463","reported_seq":10603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.723669+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:27.723669+0000","last_peered":"2024-09-05T13:16:27.723669+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:27.723669+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42807779,"num_objects":10452,"num_object_clones":0,"num_object_copies":20904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":28524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5887},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":677528035,"num_objects":165413,"num_object_clones":0,"num_object_copies":330826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4107,"num_objects_unfound":0,"num_objects_dirty":165413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121212,"num_write_kb":484848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4720,"num_bytes_recovered":19324928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1378762752,"data_stored":1378679653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744004,"kb_used_data":473520,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93627836,"statfs":{"total":96636764160,"available":95874904064,"internally_reserved":0,"allocated":484884480,"data_stored":484628995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":913560,"kb_used_data":566792,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93458280,"statfs":{"total":96636764160,"available":95701278720,"internally_reserved":0,"allocated":580395008,"data_stored":580135446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539220,"kb_used_data":304836,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93832620,"statfs":{"total":96636764160,"available":96084602880,"internally_reserved":0,"allocated":312152064,"data_stored":311820242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239988399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":580857856,"data_stored":580854243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312913920,"data_stored":312838047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":484990976,"data_stored":484987363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:32.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:33.041 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:33.042 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:33.066 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-05T13:16:31.599960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":677528054,"num_objects":165415,"num_object_clones":0,"num_object_copies":330830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4107,"num_objects_unfound":0,"num_objects_dirty":165415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121214,"num_write_kb":484850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4721,"num_bytes_recovered":19324947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2196784,"kb_used_data":1345148,"kb_used_omap":12,"kb_used_meta":851571,"kb_avail":280918736,"statfs":{"total":289910292480,"available":287660785664,"internally_reserved":0,"allocated":1377431552,"data_stored":1376584683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":872008710},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,5,2,11,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17772544,"num_objects":4339,"num_object_clones":0,"num_object_copies":8678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-155,"num_objects_unfound":0,"num_objects_dirty":4339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":634880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004261"},"pg_stats":[{"pgid":"2.f","version":"45'10332","reported_seq":10406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.768482+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:27.768482+0000","last_peered":"2024-09-05T13:16:27.768482+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:27.768482+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10332,"num_write_kb":41328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10250","reported_seq":10354,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.746101+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:27.746101+0000","last_peered":"2024-09-05T13:16:27.746101+0000","last_clean":"2024-09-05T13:16:27.746101+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:27.746101+0000","last_undegraded":"2024-09-05T13:16:27.746101+0000","last_fullsized":"2024-09-05T13:16:27.746101+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41959424,"num_objects":10244,"num_object_clones":0,"num_object_copies":20488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8260,"num_write_kb":33040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10472","reported_seq":10599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.905956+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:27.905956+0000","last_peered":"2024-09-05T13:16:27.905956+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:27.905956+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42876928,"num_objects":10468,"num_object_clones":0,"num_object_copies":20936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8283,"num_write_kb":33132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":5976}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10362","reported_seq":10431,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.843157+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:27.843157+0000","last_peered":"2024-09-05T13:16:27.843157+0000","last_clean":"2024-09-05T13:16:27.843157+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:27.843157+0000","last_undegraded":"2024-09-05T13:16:27.843157+0000","last_fullsized":"2024-09-05T13:16:27.843157+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42442752,"num_objects":10362,"num_object_clones":0,"num_object_copies":20724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10362,"num_write_kb":41448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10302","reported_seq":10436,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.693663+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:27.693663+0000","last_peered":"2024-09-05T13:16:27.693663+0000","last_clean":"2024-09-05T13:16:27.693663+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:27.693663+0000","last_undegraded":"2024-09-05T13:16:27.693663+0000","last_fullsized":"2024-09-05T13:16:27.693663+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42168320,"num_objects":10295,"num_object_clones":0,"num_object_copies":20590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10432","reported_seq":10567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.811097+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:27.811097+0000","last_peered":"2024-09-05T13:16:27.811097+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:27.811097+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42708992,"num_objects":10427,"num_object_clones":0,"num_object_copies":20854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6887,"num_write_kb":27548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5976},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10180","reported_seq":10976,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.744446+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:27.744446+0000","last_peered":"2024-09-05T13:16:27.744446+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:27.744446+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41672704,"num_objects":10174,"num_object_clones":0,"num_object_copies":20348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4107,"num_objects_unfound":0,"num_objects_dirty":10174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6499,"num_write_kb":25996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1343488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6066},{"shards":"0,2","objects":4107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10412","reported_seq":10544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.685056+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:27.685056+0000","last_peered":"2024-09-05T13:16:27.685056+0000","last_clean":"2024-09-05T13:16:27.685056+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:27.685056+0000","last_undegraded":"2024-09-05T13:16:27.685056+0000","last_fullsized":"2024-09-05T13:16:27.685056+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42639360,"num_objects":10410,"num_object_clones":0,"num_object_copies":20820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":28088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10392","reported_seq":10526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.883345+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:27.883345+0000","last_peered":"2024-09-05T13:16:27.883345+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:27.883345+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42536960,"num_objects":10385,"num_object_clones":0,"num_object_copies":20770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":32184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":5934}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10316","reported_seq":10454,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.809612+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:27.809612+0000","last_peered":"2024-09-05T13:16:27.809612+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:27.809612+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42221568,"num_objects":10308,"num_object_clones":0,"num_object_copies":20616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6683,"num_write_kb":26732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5854},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10320","reported_seq":19220,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.723992+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:27.723992+0000","last_peered":"2024-09-05T13:16:27.723992+0000","last_clean":"2024-09-05T13:16:27.723992+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:27.723992+0000","last_undegraded":"2024-09-05T13:16:27.723992+0000","last_fullsized":"2024-09-05T13:16:27.723992+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42250240,"num_objects":10315,"num_object_clones":0,"num_object_copies":20630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7915,"num_write_kb":31660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10164","reported_seq":10287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.839738+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:27.839738+0000","last_peered":"2024-09-05T13:16:27.839738+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:27.839738+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41607168,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7673,"num_write_kb":30692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5771}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10297","reported_seq":10435,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.748467+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:27.748467+0000","last_peered":"2024-09-05T13:16:27.748467+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:27.748467+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42168320,"num_objects":10295,"num_object_clones":0,"num_object_copies":20590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":25792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5876},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10396","reported_seq":10582,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.781258+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:27.781258+0000","last_peered":"2024-09-05T13:16:27.781258+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:27.781258+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42549248,"num_objects":10388,"num_object_clones":0,"num_object_copies":20776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6295,"num_write_kb":25180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5944},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10404","reported_seq":10543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.676982+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:27.676982+0000","last_peered":"2024-09-05T13:16:27.676982+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:27.676982+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42598400,"num_objects":10400,"num_object_clones":0,"num_object_copies":20800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6534,"num_write_kb":26136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5834},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10463","reported_seq":10603,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:27.723669+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:27.723669+0000","last_peered":"2024-09-05T13:16:27.723669+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:27.723669+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42807779,"num_objects":10452,"num_object_clones":0,"num_object_copies":20904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":28524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5887},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":677528035,"num_objects":165413,"num_object_clones":0,"num_object_copies":330826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4107,"num_objects_unfound":0,"num_objects_dirty":165413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121212,"num_write_kb":484848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4720,"num_bytes_recovered":19324928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1378762752,"data_stored":1378679653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744004,"kb_used_data":473520,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93627836,"statfs":{"total":96636764160,"available":95874904064,"internally_reserved":0,"allocated":484884480,"data_stored":484628995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":276950701},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":913560,"kb_used_data":566792,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93458280,"statfs":{"total":96636764160,"available":95701278720,"internally_reserved":0,"allocated":580395008,"data_stored":580135446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539220,"kb_used_data":304836,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93832620,"statfs":{"total":96636764160,"available":96084602880,"internally_reserved":0,"allocated":312152064,"data_stored":311820242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239988399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":580857856,"data_stored":580854243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312913920,"data_stored":312838047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":484990976,"data_stored":484987363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:33.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:33.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10332", 'reported_seq': 10406, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.768482+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:27.768482+0000', 'last_peered': '2024-09-05T13:16:27.768482+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:27.768482+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42319872, 'num_objects': 10332, 'num_object_clones': 0, 'num_object_copies': 20664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10332, 'num_write_kb': 41328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5911}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:33.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:33.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10472", 'reported_seq': 10599, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.905956+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:27.905956+0000', 'last_peered': '2024-09-05T13:16:27.905956+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:27.905956+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42876928, 'num_objects': 10468, 'num_object_clones': 0, 'num_object_copies': 20936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10468, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8283, 'num_write_kb': 33132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 5976}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:33.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:33.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10432", 'reported_seq': 10567, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.811097+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:27.811097+0000', 'last_peered': '2024-09-05T13:16:27.811097+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:27.811097+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42708992, 'num_objects': 10427, 'num_object_clones': 0, 'num_object_copies': 20854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10427, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6887, 'num_write_kb': 27548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5976}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:33.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:33.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10180", 'reported_seq': 10976, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.744446+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:27.744446+0000', 'last_peered': '2024-09-05T13:16:27.744446+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:27.744446+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41672704, 'num_objects': 10174, 'num_object_clones': 0, 'num_object_copies': 20348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 4107, 'num_objects_unfound': 0, 'num_objects_dirty': 10174, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6499, 'num_write_kb': 25996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 328, 'num_bytes_recovered': 1343488, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6066}, {'shards': '0,2', 'objects': 4107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10392", 'reported_seq': 10526, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.883345+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:27.883345+0000', 'last_peered': '2024-09-05T13:16:27.883345+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:27.883345+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42536960, 'num_objects': 10385, 'num_object_clones': 0, 'num_object_copies': 20770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10385, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8046, 'num_write_kb': 32184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 5934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10316", 'reported_seq': 10454, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.809612+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:27.809612+0000', 'last_peered': '2024-09-05T13:16:27.809612+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:27.809612+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42221568, 'num_objects': 10308, 'num_object_clones': 0, 'num_object_copies': 20616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6683, 'num_write_kb': 26732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5854}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10164", 'reported_seq': 10287, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.839738+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:27.839738+0000', 'last_peered': '2024-09-05T13:16:27.839738+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:27.839738+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41607168, 'num_objects': 10158, 'num_object_clones': 0, 'num_object_copies': 20316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7673, 'num_write_kb': 30692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 5771}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10297", 'reported_seq': 10435, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.748467+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:27.748467+0000', 'last_peered': '2024-09-05T13:16:27.748467+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:27.748467+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42168320, 'num_objects': 10295, 'num_object_clones': 0, 'num_object_copies': 20590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6448, 'num_write_kb': 25792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5876}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10396", 'reported_seq': 10582, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.781258+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:27.781258+0000', 'last_peered': '2024-09-05T13:16:27.781258+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:27.781258+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42549248, 'num_objects': 10388, 'num_object_clones': 0, 'num_object_copies': 20776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6295, 'num_write_kb': 25180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5944}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:33.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:33.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10404", 'reported_seq': 10543, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.676982+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:27.676982+0000', 'last_peered': '2024-09-05T13:16:27.676982+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:27.676982+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42598400, 'num_objects': 10400, 'num_object_clones': 0, 'num_object_copies': 20800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10400, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6534, 'num_write_kb': 26136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5834}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:33.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:33.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10463", 'reported_seq': 10603, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:27.723669+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:27.723669+0000', 'last_peered': '2024-09-05T13:16:27.723669+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:27.723669+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42807779, 'num_objects': 10452, 'num_object_clones': 0, 'num_object_copies': 20904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7131, 'num_write_kb': 28524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5887}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:33.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:33.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Total time run: 240.047 2024-09-05T13:16:33.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Total writes made: 167692 2024-09-05T13:16:33.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Write size: 4096 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Object size: 4096 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Bandwidth (MB/sec): 2.72882 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Stddev Bandwidth: 2.14461 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Max bandwidth (MB/sec): 12.6445 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Min bandwidth (MB/sec): 0 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Average IOPS: 698 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Stddev IOPS: 549.02 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Max IOPS: 3237 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Min IOPS: 0 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Average Latency(s): 0.0286227 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Stddev Latency(s): 0.0429703 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Max latency(s): 2.69414 2024-09-05T13:16:33.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Min latency(s): 0.00201413 2024-09-05T13:16:33.261 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Cleaning up (deleting benchmark objects) 2024-09-05T13:16:33.738 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:33.739 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/330830 objects degraded (13.502%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":320,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":165415,"data_bytes":677528054,"bytes_used":2249506816,"bytes_avail":287660785664,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":330830,"degraded_ratio":0.13502100776834022,"misplaced_objects":4107,"misplaced_total":330830,"misplaced_ratio":0.01241423087386271,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":52887,"recovering_keys_per_sec":0,"num_objects_recovered":155,"num_bytes_recovered":634880,"num_keys_recovered":0,"write_bytes_sec":1480519,"read_op_per_sec":0,"write_op_per_sec":361},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:33.771 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:33.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:34.387 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:34.389 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:34.421 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-05T13:16:33.600715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":686379510,"num_objects":167576,"num_object_clones":0,"num_object_copies":335152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":167576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123375,"num_write_kb":493494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4804,"num_bytes_recovered":19664915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2199248,"kb_used_data":1355936,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":280916272,"statfs":{"total":289910292480,"available":287658262528,"internally_reserved":0,"allocated":1388478464,"data_stored":1387631595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":863489030},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,9,9,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26624000,"num_objects":6500,"num_object_clones":0,"num_object_copies":13000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-238,"num_objects_unfound":0,"num_objects_dirty":6500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6500,"num_write_kb":26000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004254"},"pg_stats":[{"pgid":"2.f","version":"45'10476","reported_seq":10551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.855884+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:32.855884+0000","last_peered":"2024-09-05T13:16:32.855884+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:32.855884+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10476,"num_write_kb":41904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6055},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10375","reported_seq":10479,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.784508+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:32.784508+0000","last_peered":"2024-09-05T13:16:32.784508+0000","last_clean":"2024-09-05T13:16:32.784508+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:32.784508+0000","last_undegraded":"2024-09-05T13:16:32.784508+0000","last_fullsized":"2024-09-05T13:16:32.784508+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42471424,"num_objects":10369,"num_object_clones":0,"num_object_copies":20738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":33540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10594","reported_seq":10721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.949074+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:32.949074+0000","last_peered":"2024-09-05T13:16:32.949074+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:32.949074+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43376640,"num_objects":10590,"num_object_clones":0,"num_object_copies":21180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8405,"num_write_kb":33620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":6098}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10508","reported_seq":10577,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.856319+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:32.856319+0000","last_peered":"2024-09-05T13:16:32.856319+0000","last_clean":"2024-09-05T13:16:32.856319+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:32.856319+0000","last_undegraded":"2024-09-05T13:16:32.856319+0000","last_fullsized":"2024-09-05T13:16:32.856319+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43040768,"num_objects":10508,"num_object_clones":0,"num_object_copies":21016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10508,"num_write_kb":42032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10438","reported_seq":10572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.827072+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:32.827072+0000","last_peered":"2024-09-05T13:16:32.827072+0000","last_clean":"2024-09-05T13:16:32.827072+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:32.827072+0000","last_undegraded":"2024-09-05T13:16:32.827072+0000","last_fullsized":"2024-09-05T13:16:32.827072+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42725376,"num_objects":10431,"num_object_clones":0,"num_object_copies":20862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10566","reported_seq":10704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.817354+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:32.817354+0000","last_peered":"2024-09-05T13:16:32.817354+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:32.817354+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43257856,"num_objects":10561,"num_object_clones":0,"num_object_copies":21122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7021,"num_write_kb":28084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6110},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10313","reported_seq":11275,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.879559+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:32.879559+0000","last_peered":"2024-09-05T13:16:32.879559+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:32.879559+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42217472,"num_objects":10307,"num_object_clones":0,"num_object_copies":20614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":10307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":1683456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6282},{"shards":"0,2","objects":4024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10546","reported_seq":10678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.728681+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:32.728681+0000","last_peered":"2024-09-05T13:16:32.728681+0000","last_clean":"2024-09-05T13:16:32.728681+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:32.728681+0000","last_undegraded":"2024-09-05T13:16:32.728681+0000","last_fullsized":"2024-09-05T13:16:32.728681+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43188224,"num_objects":10544,"num_object_clones":0,"num_object_copies":21088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7156,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10546","reported_seq":10680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.915136+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:32.915136+0000","last_peered":"2024-09-05T13:16:32.915136+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:32.915136+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43167744,"num_objects":10539,"num_object_clones":0,"num_object_copies":21078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8200,"num_write_kb":32800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":6088}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10452","reported_seq":10590,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.834212+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:32.834212+0000","last_peered":"2024-09-05T13:16:32.834212+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:32.834212+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42778624,"num_objects":10444,"num_object_clones":0,"num_object_copies":20888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6819,"num_write_kb":27276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10451","reported_seq":19351,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.929616+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:32.929616+0000","last_peered":"2024-09-05T13:16:32.929616+0000","last_clean":"2024-09-05T13:16:32.929616+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:32.929616+0000","last_undegraded":"2024-09-05T13:16:32.929616+0000","last_fullsized":"2024-09-05T13:16:32.929616+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42786816,"num_objects":10446,"num_object_clones":0,"num_object_copies":20892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":32184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10291","reported_seq":10414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.837927+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:32.837927+0000","last_peered":"2024-09-05T13:16:32.837927+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:32.837927+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42127360,"num_objects":10285,"num_object_clones":0,"num_object_copies":20570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7800,"num_write_kb":31200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10426","reported_seq":10566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.875963+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:32.875963+0000","last_peered":"2024-09-05T13:16:32.875963+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:32.875963+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42696704,"num_objects":10424,"num_object_clones":0,"num_object_copies":20848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10549","reported_seq":10736,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.891952+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:32.891952+0000","last_peered":"2024-09-05T13:16:32.891952+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:32.891952+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43175936,"num_objects":10541,"num_object_clones":0,"num_object_copies":21082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":25792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6097},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10532","reported_seq":10666,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.782424+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:32.782424+0000","last_peered":"2024-09-05T13:16:32.782424+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:32.782424+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43122688,"num_objects":10528,"num_object_clones":0,"num_object_copies":21056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6662,"num_write_kb":26648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10592","reported_seq":10732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.784350+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:32.784350+0000","last_peered":"2024-09-05T13:16:32.784350+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:32.784350+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43336163,"num_objects":10581,"num_object_clones":0,"num_object_copies":21162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":29040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6016},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":686379491,"num_objects":167574,"num_object_clones":0,"num_object_copies":335148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":167574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123373,"num_write_kb":493492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4803,"num_bytes_recovered":19664896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1389797376,"data_stored":1389714277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752504,"kb_used_data":477864,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93619336,"statfs":{"total":96636764160,"available":95866200064,"internally_reserved":0,"allocated":489332736,"data_stored":489077251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907524,"kb_used_data":573236,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93464316,"statfs":{"total":96636764160,"available":95707459584,"internally_reserved":0,"allocated":586993664,"data_stored":586734102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539220,"kb_used_data":304836,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93832620,"statfs":{"total":96636764160,"available":96084602880,"internally_reserved":0,"allocated":312152064,"data_stored":311820242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239988399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":587513856,"data_stored":587510243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312913920,"data_stored":312838047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":489369600,"data_stored":489365987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:37.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:37.977 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:37.979 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:37.996 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-05T13:16:37.602324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":686379510,"num_objects":167576,"num_object_clones":0,"num_object_copies":335152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":167576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123375,"num_write_kb":493494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4804,"num_bytes_recovered":19664915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2207380,"kb_used_data":1359904,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280908140,"statfs":{"total":289910292480,"available":287649935360,"internally_reserved":0,"allocated":1392541696,"data_stored":1391690755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":867748870},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,9,12,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17805312,"num_objects":4347,"num_object_clones":0,"num_object_copies":8694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-166,"num_objects_unfound":0,"num_objects_dirty":4347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4347,"num_write_kb":17388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004169"},"pg_stats":[{"pgid":"2.f","version":"45'10476","reported_seq":10551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.855884+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:32.855884+0000","last_peered":"2024-09-05T13:16:32.855884+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:32.855884+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10476,"num_write_kb":41904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6055},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10375","reported_seq":10479,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.784508+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:32.784508+0000","last_peered":"2024-09-05T13:16:32.784508+0000","last_clean":"2024-09-05T13:16:32.784508+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:32.784508+0000","last_undegraded":"2024-09-05T13:16:32.784508+0000","last_fullsized":"2024-09-05T13:16:32.784508+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42471424,"num_objects":10369,"num_object_clones":0,"num_object_copies":20738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":33540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10594","reported_seq":10721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.949074+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:32.949074+0000","last_peered":"2024-09-05T13:16:32.949074+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:32.949074+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43376640,"num_objects":10590,"num_object_clones":0,"num_object_copies":21180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8405,"num_write_kb":33620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":6098}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10508","reported_seq":10577,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.856319+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:32.856319+0000","last_peered":"2024-09-05T13:16:32.856319+0000","last_clean":"2024-09-05T13:16:32.856319+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:32.856319+0000","last_undegraded":"2024-09-05T13:16:32.856319+0000","last_fullsized":"2024-09-05T13:16:32.856319+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43040768,"num_objects":10508,"num_object_clones":0,"num_object_copies":21016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10508,"num_write_kb":42032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10438","reported_seq":10572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.827072+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:32.827072+0000","last_peered":"2024-09-05T13:16:32.827072+0000","last_clean":"2024-09-05T13:16:32.827072+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:32.827072+0000","last_undegraded":"2024-09-05T13:16:32.827072+0000","last_fullsized":"2024-09-05T13:16:32.827072+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42725376,"num_objects":10431,"num_object_clones":0,"num_object_copies":20862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10566","reported_seq":10704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.817354+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:32.817354+0000","last_peered":"2024-09-05T13:16:32.817354+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:32.817354+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43257856,"num_objects":10561,"num_object_clones":0,"num_object_copies":21122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7021,"num_write_kb":28084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6110},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10313","reported_seq":11275,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.879559+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:32.879559+0000","last_peered":"2024-09-05T13:16:32.879559+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:32.879559+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42217472,"num_objects":10307,"num_object_clones":0,"num_object_copies":20614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":10307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":1683456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6282},{"shards":"0,2","objects":4024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10546","reported_seq":10678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.728681+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:32.728681+0000","last_peered":"2024-09-05T13:16:32.728681+0000","last_clean":"2024-09-05T13:16:32.728681+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:32.728681+0000","last_undegraded":"2024-09-05T13:16:32.728681+0000","last_fullsized":"2024-09-05T13:16:32.728681+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43188224,"num_objects":10544,"num_object_clones":0,"num_object_copies":21088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7156,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10546","reported_seq":10680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.915136+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:32.915136+0000","last_peered":"2024-09-05T13:16:32.915136+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:32.915136+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43167744,"num_objects":10539,"num_object_clones":0,"num_object_copies":21078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8200,"num_write_kb":32800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":6088}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10452","reported_seq":10590,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.834212+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:32.834212+0000","last_peered":"2024-09-05T13:16:32.834212+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:32.834212+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42778624,"num_objects":10444,"num_object_clones":0,"num_object_copies":20888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6819,"num_write_kb":27276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10451","reported_seq":19351,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.929616+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:32.929616+0000","last_peered":"2024-09-05T13:16:32.929616+0000","last_clean":"2024-09-05T13:16:32.929616+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:32.929616+0000","last_undegraded":"2024-09-05T13:16:32.929616+0000","last_fullsized":"2024-09-05T13:16:32.929616+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42786816,"num_objects":10446,"num_object_clones":0,"num_object_copies":20892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":32184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10291","reported_seq":10414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.837927+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:32.837927+0000","last_peered":"2024-09-05T13:16:32.837927+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:32.837927+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42127360,"num_objects":10285,"num_object_clones":0,"num_object_copies":20570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7800,"num_write_kb":31200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10426","reported_seq":10566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.875963+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:32.875963+0000","last_peered":"2024-09-05T13:16:32.875963+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:32.875963+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42696704,"num_objects":10424,"num_object_clones":0,"num_object_copies":20848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10549","reported_seq":10736,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.891952+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:32.891952+0000","last_peered":"2024-09-05T13:16:32.891952+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:32.891952+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43175936,"num_objects":10541,"num_object_clones":0,"num_object_copies":21082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":25792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6097},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10532","reported_seq":10666,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.782424+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:32.782424+0000","last_peered":"2024-09-05T13:16:32.782424+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:32.782424+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43122688,"num_objects":10528,"num_object_clones":0,"num_object_copies":21056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6662,"num_write_kb":26648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10592","reported_seq":10732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.784350+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:32.784350+0000","last_peered":"2024-09-05T13:16:32.784350+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:32.784350+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43336163,"num_objects":10581,"num_object_clones":0,"num_object_copies":21162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":29040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6016},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":686379491,"num_objects":167574,"num_object_clones":0,"num_object_copies":335148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":167574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123373,"num_write_kb":493492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4803,"num_bytes_recovered":19664896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1392648192,"data_stored":1392561021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752504,"kb_used_data":477864,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93619336,"statfs":{"total":96636764160,"available":95866200064,"internally_reserved":0,"allocated":489332736,"data_stored":489077251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907524,"kb_used_data":573236,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93464316,"statfs":{"total":96636764160,"available":95707459584,"internally_reserved":0,"allocated":586993664,"data_stored":586734102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547352,"kb_used_data":308804,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93824488,"statfs":{"total":96636764160,"available":96076275712,"internally_reserved":0,"allocated":316215296,"data_stored":315879402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":587513856,"data_stored":587510243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315764736,"data_stored":315684791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":489369600,"data_stored":489365987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:37.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:38.574 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:38.574 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:38.599 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-05T13:16:37.602324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":686379510,"num_objects":167576,"num_object_clones":0,"num_object_copies":335152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":167576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123375,"num_write_kb":493494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4804,"num_bytes_recovered":19664915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2207380,"kb_used_data":1359904,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280908140,"statfs":{"total":289910292480,"available":287649935360,"internally_reserved":0,"allocated":1392541696,"data_stored":1391690755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":867748870},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,9,12,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17805312,"num_objects":4347,"num_object_clones":0,"num_object_copies":8694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-166,"num_objects_unfound":0,"num_objects_dirty":4347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4347,"num_write_kb":17388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004169"},"pg_stats":[{"pgid":"2.f","version":"45'10476","reported_seq":10551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.855884+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:32.855884+0000","last_peered":"2024-09-05T13:16:32.855884+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:32.855884+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10476,"num_write_kb":41904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6055},{"shards":"0,2","objects":4420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10375","reported_seq":10479,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.784508+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:32.784508+0000","last_peered":"2024-09-05T13:16:32.784508+0000","last_clean":"2024-09-05T13:16:32.784508+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:32.784508+0000","last_undegraded":"2024-09-05T13:16:32.784508+0000","last_fullsized":"2024-09-05T13:16:32.784508+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42471424,"num_objects":10369,"num_object_clones":0,"num_object_copies":20738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":33540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10594","reported_seq":10721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.949074+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:32.949074+0000","last_peered":"2024-09-05T13:16:32.949074+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:32.949074+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43376640,"num_objects":10590,"num_object_clones":0,"num_object_copies":21180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8405,"num_write_kb":33620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4492},{"shards":"2","objects":6098}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10508","reported_seq":10577,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.856319+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:32.856319+0000","last_peered":"2024-09-05T13:16:32.856319+0000","last_clean":"2024-09-05T13:16:32.856319+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:32.856319+0000","last_undegraded":"2024-09-05T13:16:32.856319+0000","last_fullsized":"2024-09-05T13:16:32.856319+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43040768,"num_objects":10508,"num_object_clones":0,"num_object_copies":21016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10508,"num_write_kb":42032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10438","reported_seq":10572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.827072+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:32.827072+0000","last_peered":"2024-09-05T13:16:32.827072+0000","last_clean":"2024-09-05T13:16:32.827072+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:32.827072+0000","last_undegraded":"2024-09-05T13:16:32.827072+0000","last_fullsized":"2024-09-05T13:16:32.827072+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42725376,"num_objects":10431,"num_object_clones":0,"num_object_copies":20862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10566","reported_seq":10704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.817354+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:32.817354+0000","last_peered":"2024-09-05T13:16:32.817354+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:32.817354+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43257856,"num_objects":10561,"num_object_clones":0,"num_object_copies":21122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7021,"num_write_kb":28084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6110},{"shards":"0,2","objects":4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10313","reported_seq":11275,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.879559+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:32.879559+0000","last_peered":"2024-09-05T13:16:32.879559+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:32.879559+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42217472,"num_objects":10307,"num_object_clones":0,"num_object_copies":20614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":10307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6632,"num_write_kb":26528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":1683456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6282},{"shards":"0,2","objects":4024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10546","reported_seq":10678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.728681+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:32.728681+0000","last_peered":"2024-09-05T13:16:32.728681+0000","last_clean":"2024-09-05T13:16:32.728681+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:32.728681+0000","last_undegraded":"2024-09-05T13:16:32.728681+0000","last_fullsized":"2024-09-05T13:16:32.728681+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43188224,"num_objects":10544,"num_object_clones":0,"num_object_copies":21088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7156,"num_write_kb":28624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10546","reported_seq":10680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.915136+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:32.915136+0000","last_peered":"2024-09-05T13:16:32.915136+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:32.915136+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43167744,"num_objects":10539,"num_object_clones":0,"num_object_copies":21078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8200,"num_write_kb":32800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4451},{"shards":"2","objects":6088}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10452","reported_seq":10590,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.834212+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:32.834212+0000","last_peered":"2024-09-05T13:16:32.834212+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:32.834212+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42778624,"num_objects":10444,"num_object_clones":0,"num_object_copies":20888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6819,"num_write_kb":27276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5990},{"shards":"0,2","objects":4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10451","reported_seq":19351,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:32.929616+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:32.929616+0000","last_peered":"2024-09-05T13:16:32.929616+0000","last_clean":"2024-09-05T13:16:32.929616+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:32.929616+0000","last_undegraded":"2024-09-05T13:16:32.929616+0000","last_fullsized":"2024-09-05T13:16:32.929616+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42786816,"num_objects":10446,"num_object_clones":0,"num_object_copies":20892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":32184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10291","reported_seq":10414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.837927+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:32.837927+0000","last_peered":"2024-09-05T13:16:32.837927+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:32.837927+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42127360,"num_objects":10285,"num_object_clones":0,"num_object_copies":20570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7800,"num_write_kb":31200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4387},{"shards":"2","objects":5898}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10426","reported_seq":10566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.875963+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:32.875963+0000","last_peered":"2024-09-05T13:16:32.875963+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:32.875963+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42696704,"num_objects":10424,"num_object_clones":0,"num_object_copies":20848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6005},{"shards":"0,2","objects":4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10549","reported_seq":10736,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.891952+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:32.891952+0000","last_peered":"2024-09-05T13:16:32.891952+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:32.891952+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43175936,"num_objects":10541,"num_object_clones":0,"num_object_copies":21082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":25792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6097},{"shards":"0,2","objects":4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10532","reported_seq":10666,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.782424+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:32.782424+0000","last_peered":"2024-09-05T13:16:32.782424+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:32.782424+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43122688,"num_objects":10528,"num_object_clones":0,"num_object_copies":21056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6662,"num_write_kb":26648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10592","reported_seq":10732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:32.784350+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:32.784350+0000","last_peered":"2024-09-05T13:16:32.784350+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:32.784350+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43336163,"num_objects":10581,"num_object_clones":0,"num_object_copies":21162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7260,"num_write_kb":29040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6016},{"shards":"0,2","objects":4565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":686379491,"num_objects":167574,"num_object_clones":0,"num_object_copies":335148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":4024,"num_objects_unfound":0,"num_objects_dirty":167574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123373,"num_write_kb":493492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4803,"num_bytes_recovered":19664896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1392648192,"data_stored":1392561021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752504,"kb_used_data":477864,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93619336,"statfs":{"total":96636764160,"available":95866200064,"internally_reserved":0,"allocated":489332736,"data_stored":489077251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":281210541},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907524,"kb_used_data":573236,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93464316,"statfs":{"total":96636764160,"available":95707459584,"internally_reserved":0,"allocated":586993664,"data_stored":586734102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547352,"kb_used_data":308804,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93824488,"statfs":{"total":96636764160,"available":96076275712,"internally_reserved":0,"allocated":316215296,"data_stored":315879402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":587513856,"data_stored":587510243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315764736,"data_stored":315684791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":489369600,"data_stored":489365987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:38.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:38.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10476", 'reported_seq': 10551, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.855884+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:32.855884+0000', 'last_peered': '2024-09-05T13:16:32.855884+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:32.855884+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42909696, 'num_objects': 10476, 'num_object_clones': 0, 'num_object_copies': 20952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10476, 'num_write_kb': 41904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6055}, {'shards': '0,2', 'objects': 4420}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:38.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:38.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10594", 'reported_seq': 10721, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.949074+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:32.949074+0000', 'last_peered': '2024-09-05T13:16:32.949074+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:32.949074+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43376640, 'num_objects': 10590, 'num_object_clones': 0, 'num_object_copies': 21180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8405, 'num_write_kb': 33620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4492}, {'shards': '2', 'objects': 6098}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:38.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10566", 'reported_seq': 10704, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.817354+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:32.817354+0000', 'last_peered': '2024-09-05T13:16:32.817354+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:32.817354+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43257856, 'num_objects': 10561, 'num_object_clones': 0, 'num_object_copies': 21122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7021, 'num_write_kb': 28084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6110}, {'shards': '0,2', 'objects': 4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10313", 'reported_seq': 11275, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.879559+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:32.879559+0000', 'last_peered': '2024-09-05T13:16:32.879559+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:32.879559+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42217472, 'num_objects': 10307, 'num_object_clones': 0, 'num_object_copies': 20614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 4024, 'num_objects_unfound': 0, 'num_objects_dirty': 10307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6632, 'num_write_kb': 26528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 411, 'num_bytes_recovered': 1683456, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6282}, {'shards': '0,2', 'objects': 4024}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10546", 'reported_seq': 10680, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.915136+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:32.915136+0000', 'last_peered': '2024-09-05T13:16:32.915136+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:32.915136+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43167744, 'num_objects': 10539, 'num_object_clones': 0, 'num_object_copies': 21078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10539, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8200, 'num_write_kb': 32800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4451}, {'shards': '2', 'objects': 6088}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10452", 'reported_seq': 10590, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.834212+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:32.834212+0000', 'last_peered': '2024-09-05T13:16:32.834212+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:32.834212+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42778624, 'num_objects': 10444, 'num_object_clones': 0, 'num_object_copies': 20888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6819, 'num_write_kb': 27276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5990}, {'shards': '0,2', 'objects': 4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:38.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10291", 'reported_seq': 10414, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.837927+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:32.837927+0000', 'last_peered': '2024-09-05T13:16:32.837927+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:32.837927+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42127360, 'num_objects': 10285, 'num_object_clones': 0, 'num_object_copies': 20570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7800, 'num_write_kb': 31200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4387}, {'shards': '2', 'objects': 5898}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:38.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:38.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10426", 'reported_seq': 10566, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.875963+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:32.875963+0000', 'last_peered': '2024-09-05T13:16:32.875963+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:32.875963+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42696704, 'num_objects': 10424, 'num_object_clones': 0, 'num_object_copies': 20848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10424, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6577, 'num_write_kb': 26308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6005}, {'shards': '0,2', 'objects': 4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:38.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:38.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10549", 'reported_seq': 10736, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.891952+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:32.891952+0000', 'last_peered': '2024-09-05T13:16:32.891952+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:32.891952+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43175936, 'num_objects': 10541, 'num_object_clones': 0, 'num_object_copies': 21082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6448, 'num_write_kb': 25792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6097}, {'shards': '0,2', 'objects': 4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:38.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:38.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10532", 'reported_seq': 10666, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.782424+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:32.782424+0000', 'last_peered': '2024-09-05T13:16:32.782424+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:32.782424+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43122688, 'num_objects': 10528, 'num_object_clones': 0, 'num_object_copies': 21056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6662, 'num_write_kb': 26648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5962}, {'shards': '0,2', 'objects': 4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:38.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:38.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10592", 'reported_seq': 10732, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:32.784350+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:32.784350+0000', 'last_peered': '2024-09-05T13:16:32.784350+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:32.784350+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43336163, 'num_objects': 10581, 'num_object_clones': 0, 'num_object_copies': 21162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7260, 'num_write_kb': 29040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6016}, {'shards': '0,2', 'objects': 4565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:38.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:39.315 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:39.316 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/335152 objects degraded (13.328%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":326,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":167576,"data_bytes":686379510,"bytes_used":2260357120,"bytes_avail":287649935360,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":335152,"degraded_ratio":0.13327982527330884,"misplaced_objects":4024,"misplaced_total":335152,"misplaced_ratio":0.012006492576502601,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":56641,"recovering_keys_per_sec":0,"num_objects_recovered":166,"num_bytes_recovered":679936,"num_keys_recovered":0,"write_bytes_sec":1483260,"read_op_per_sec":0,"write_op_per_sec":362},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:39.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:39.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:39.894 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:39.916 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-05T13:16:39.603142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":678453774,"num_objects":165642,"num_object_clones":0,"num_object_copies":331284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3934,"num_objects_unfound":0,"num_objects_dirty":165642,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125549,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4894,"num_bytes_recovered":20033555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2191932,"kb_used_data":1352768,"kb_used_omap":12,"kb_used_meta":839091,"kb_avail":280923588,"statfs":{"total":289910292480,"available":287665754112,"internally_reserved":0,"allocated":1385234432,"data_stored":1384379419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":859229190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,7,10,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9879576,"num_objects":2413,"num_object_clones":0,"num_object_copies":4826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":2413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6521,"num_write_kb":17865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004329"},"pg_stats":[{"pgid":"2.f","version":"45'10610","reported_seq":10685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.964652+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:37.964652+0000","last_peered":"2024-09-05T13:16:37.964652+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:37.964652+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42442752,"num_objects":10362,"num_object_clones":0,"num_object_copies":20724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5941},{"shards":"0,2","objects":4296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10521","reported_seq":10625,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.926987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:37.926987+0000","last_peered":"2024-09-05T13:16:37.926987+0000","last_clean":"2024-09-05T13:16:37.926987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:37.926987+0000","last_undegraded":"2024-09-05T13:16:37.926987+0000","last_fullsized":"2024-09-05T13:16:37.926987+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41914368,"num_objects":10233,"num_object_clones":0,"num_object_copies":20466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8531,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10726","reported_seq":10854,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.849486+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:37.849486+0000","last_peered":"2024-09-05T13:16:37.849486+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:37.849486+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42881048,"num_objects":10470,"num_object_clones":0,"num_object_copies":20940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10470,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8537,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4366},{"shards":"2","objects":5978}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10644","reported_seq":10713,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.877308+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:37.877308+0000","last_peered":"2024-09-05T13:16:37.877308+0000","last_clean":"2024-09-05T13:16:37.877308+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:37.877308+0000","last_undegraded":"2024-09-05T13:16:37.877308+0000","last_fullsized":"2024-09-05T13:16:37.877308+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42524672,"num_objects":10382,"num_object_clones":0,"num_object_copies":20764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10644,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10600","reported_seq":10734,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.828234+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:37.828234+0000","last_peered":"2024-09-05T13:16:37.828234+0000","last_clean":"2024-09-05T13:16:37.828234+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:37.828234+0000","last_undegraded":"2024-09-05T13:16:37.828234+0000","last_fullsized":"2024-09-05T13:16:37.828234+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42119168,"num_objects":10283,"num_object_clones":0,"num_object_copies":20566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7140,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10697","reported_seq":10835,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.841248+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:37.841248+0000","last_peered":"2024-09-05T13:16:37.841248+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:37.841248+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42835968,"num_objects":10458,"num_object_clones":0,"num_object_copies":20916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7152,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6007},{"shards":"0,2","objects":4334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10431","reported_seq":11574,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.912880+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:37.912880+0000","last_peered":"2024-09-05T13:16:37.912880+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:37.912880+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41766912,"num_objects":10197,"num_object_clones":0,"num_object_copies":20394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3934,"num_objects_unfound":0,"num_objects_dirty":10197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6750,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":2052096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6262},{"shards":"0,2","objects":3934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10678","reported_seq":10810,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.910357+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:37.910357+0000","last_peered":"2024-09-05T13:16:37.910357+0000","last_clean":"2024-09-05T13:16:37.910357+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:37.910357+0000","last_undegraded":"2024-09-05T13:16:37.910357+0000","last_fullsized":"2024-09-05T13:16:37.910357+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42704896,"num_objects":10426,"num_object_clones":0,"num_object_copies":20852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7288,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10687","reported_seq":10820,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.950100+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:37.950100+0000","last_peered":"2024-09-05T13:16:37.950100+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:37.950100+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42622976,"num_objects":10406,"num_object_clones":0,"num_object_copies":20812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8341,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4314},{"shards":"2","objects":5955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10592","reported_seq":10730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.845205+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:37.845205+0000","last_peered":"2024-09-05T13:16:37.845205+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:37.845205+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42262528,"num_objects":10318,"num_object_clones":0,"num_object_copies":20636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6959,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5864},{"shards":"0,2","objects":4321}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10585","reported_seq":19486,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.935830+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:37.935830+0000","last_peered":"2024-09-05T13:16:37.935830+0000","last_clean":"2024-09-05T13:16:37.935830+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:37.935830+0000","last_undegraded":"2024-09-05T13:16:37.935830+0000","last_fullsized":"2024-09-05T13:16:37.935830+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42295296,"num_objects":10326,"num_object_clones":0,"num_object_copies":20652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8180,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10425","reported_seq":10548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.961976+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:37.961976+0000","last_peered":"2024-09-05T13:16:37.961976+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:37.961976+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41603072,"num_objects":10157,"num_object_clones":0,"num_object_copies":20314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7934,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4256},{"shards":"2","objects":5770}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10562","reported_seq":10702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.966664+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:37.966664+0000","last_peered":"2024-09-05T13:16:37.966664+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:37.966664+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42254336,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6713,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":4297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10686","reported_seq":10873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.884609+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:37.884609+0000","last_peered":"2024-09-05T13:16:37.884609+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:37.884609+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42672128,"num_objects":10418,"num_object_clones":0,"num_object_copies":20836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6585,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5974},{"shards":"0,2","objects":4314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10668","reported_seq":10807,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.833724+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:37.833724+0000","last_peered":"2024-09-05T13:16:37.833724+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:37.833724+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42647552,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6798,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":4440}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10717","reported_seq":10857,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.899423+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:37.899423+0000","last_peered":"2024-09-05T13:16:37.899423+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:37.899423+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42906083,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7385,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":4450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":678453755,"num_objects":165640,"num_object_clones":0,"num_object_copies":331280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3934,"num_objects_unfound":0,"num_objects_dirty":165640,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125547,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4893,"num_bytes_recovered":20033536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1382432768,"data_stored":1382341525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736732,"kb_used_data":474572,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93635108,"statfs":{"total":96636764160,"available":95882350592,"internally_reserved":0,"allocated":485961728,"data_stored":485702171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907848,"kb_used_data":569392,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93463992,"statfs":{"total":96636764160,"available":95707127808,"internally_reserved":0,"allocated":583057408,"data_stored":582797846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547352,"kb_used_data":308804,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93824488,"statfs":{"total":96636764160,"available":96076275712,"internally_reserved":0,"allocated":316215296,"data_stored":315879402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":581578752,"data_stored":581575139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315764736,"data_stored":315684791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":485089280,"data_stored":485081595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:42.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:43.465 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:43.466 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:43.498 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-05T13:16:41.603939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":678453774,"num_objects":165642,"num_object_clones":0,"num_object_copies":331284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3934,"num_objects_unfound":0,"num_objects_dirty":165642,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125549,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4894,"num_bytes_recovered":20033555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2178688,"kb_used_data":1352004,"kb_used_omap":12,"kb_used_meta":826611,"kb_avail":280936832,"statfs":{"total":289910292480,"available":287679315968,"internally_reserved":0,"allocated":1384452096,"data_stored":1383597083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":846449670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,7,10,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":925720,"num_objects":227,"num_object_clones":0,"num_object_copies":454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4335,"num_write_kb":9121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004648"},"pg_stats":[{"pgid":"2.f","version":"45'10610","reported_seq":10685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.964652+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:37.964652+0000","last_peered":"2024-09-05T13:16:37.964652+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:37.964652+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42442752,"num_objects":10362,"num_object_clones":0,"num_object_copies":20724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5941},{"shards":"0,2","objects":4296}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10521","reported_seq":10625,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.926987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:37.926987+0000","last_peered":"2024-09-05T13:16:37.926987+0000","last_clean":"2024-09-05T13:16:37.926987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:37.926987+0000","last_undegraded":"2024-09-05T13:16:37.926987+0000","last_fullsized":"2024-09-05T13:16:37.926987+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41914368,"num_objects":10233,"num_object_clones":0,"num_object_copies":20466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8531,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10726","reported_seq":10854,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.849486+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:37.849486+0000","last_peered":"2024-09-05T13:16:37.849486+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:37.849486+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42881048,"num_objects":10470,"num_object_clones":0,"num_object_copies":20940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10470,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8537,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4366},{"shards":"2","objects":5978}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10644","reported_seq":10713,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.877308+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:37.877308+0000","last_peered":"2024-09-05T13:16:37.877308+0000","last_clean":"2024-09-05T13:16:37.877308+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:37.877308+0000","last_undegraded":"2024-09-05T13:16:37.877308+0000","last_fullsized":"2024-09-05T13:16:37.877308+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42524672,"num_objects":10382,"num_object_clones":0,"num_object_copies":20764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10644,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10600","reported_seq":10734,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.828234+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:37.828234+0000","last_peered":"2024-09-05T13:16:37.828234+0000","last_clean":"2024-09-05T13:16:37.828234+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:37.828234+0000","last_undegraded":"2024-09-05T13:16:37.828234+0000","last_fullsized":"2024-09-05T13:16:37.828234+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42119168,"num_objects":10283,"num_object_clones":0,"num_object_copies":20566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7140,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10697","reported_seq":10835,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.841248+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:37.841248+0000","last_peered":"2024-09-05T13:16:37.841248+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:37.841248+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42835968,"num_objects":10458,"num_object_clones":0,"num_object_copies":20916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7152,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6007},{"shards":"0,2","objects":4334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10431","reported_seq":11574,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.912880+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:37.912880+0000","last_peered":"2024-09-05T13:16:37.912880+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:37.912880+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41766912,"num_objects":10197,"num_object_clones":0,"num_object_copies":20394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3934,"num_objects_unfound":0,"num_objects_dirty":10197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6750,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":2052096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6262},{"shards":"0,2","objects":3934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10678","reported_seq":10810,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.910357+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:37.910357+0000","last_peered":"2024-09-05T13:16:37.910357+0000","last_clean":"2024-09-05T13:16:37.910357+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:37.910357+0000","last_undegraded":"2024-09-05T13:16:37.910357+0000","last_fullsized":"2024-09-05T13:16:37.910357+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42704896,"num_objects":10426,"num_object_clones":0,"num_object_copies":20852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7288,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10687","reported_seq":10820,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.950100+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:37.950100+0000","last_peered":"2024-09-05T13:16:37.950100+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:37.950100+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42622976,"num_objects":10406,"num_object_clones":0,"num_object_copies":20812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8341,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4314},{"shards":"2","objects":5955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10592","reported_seq":10730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.845205+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:37.845205+0000","last_peered":"2024-09-05T13:16:37.845205+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:37.845205+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42262528,"num_objects":10318,"num_object_clones":0,"num_object_copies":20636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6959,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5864},{"shards":"0,2","objects":4321}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10585","reported_seq":19486,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:37.935830+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:37.935830+0000","last_peered":"2024-09-05T13:16:37.935830+0000","last_clean":"2024-09-05T13:16:37.935830+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:37.935830+0000","last_undegraded":"2024-09-05T13:16:37.935830+0000","last_fullsized":"2024-09-05T13:16:37.935830+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42295296,"num_objects":10326,"num_object_clones":0,"num_object_copies":20652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8180,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10425","reported_seq":10548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.961976+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:37.961976+0000","last_peered":"2024-09-05T13:16:37.961976+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:37.961976+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41603072,"num_objects":10157,"num_object_clones":0,"num_object_copies":20314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7934,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4256},{"shards":"2","objects":5770}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10562","reported_seq":10702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.966664+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:37.966664+0000","last_peered":"2024-09-05T13:16:37.966664+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:37.966664+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42254336,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6713,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5897},{"shards":"0,2","objects":4297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10686","reported_seq":10873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.884609+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:37.884609+0000","last_peered":"2024-09-05T13:16:37.884609+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:37.884609+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42672128,"num_objects":10418,"num_object_clones":0,"num_object_copies":20836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6585,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5974},{"shards":"0,2","objects":4314}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10668","reported_seq":10807,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.833724+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:37.833724+0000","last_peered":"2024-09-05T13:16:37.833724+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:37.833724+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42647552,"num_objects":10412,"num_object_clones":0,"num_object_copies":20824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6798,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":4440}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10717","reported_seq":10857,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:37.899423+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:37.899423+0000","last_peered":"2024-09-05T13:16:37.899423+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:37.899423+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42906083,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7385,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":4450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":678453755,"num_objects":165640,"num_object_clones":0,"num_object_copies":331280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3934,"num_objects_unfound":0,"num_objects_dirty":165640,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125547,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4893,"num_bytes_recovered":20033536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1381605376,"data_stored":1381514133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736732,"kb_used_data":474572,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93635108,"statfs":{"total":96636764160,"available":95882350592,"internally_reserved":0,"allocated":485961728,"data_stored":485702171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":268431021},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907848,"kb_used_data":569392,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93463992,"statfs":{"total":96636764160,"available":95707127808,"internally_reserved":0,"allocated":583057408,"data_stored":582797846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534108,"kb_used_data":308040,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93837732,"statfs":{"total":96636764160,"available":96089837568,"internally_reserved":0,"allocated":315432960,"data_stored":315097066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":231468719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":581578752,"data_stored":581575139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314937344,"data_stored":314857399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":485089280,"data_stored":485081595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:43.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:44.066 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:44.066 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:44.091 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-05T13:16:43.604727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669426190,"num_objects":163438,"num_object_clones":0,"num_object_copies":326876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":163438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127753,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4977,"num_bytes_recovered":20373523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2198380,"kb_used_data":1341176,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":280917140,"statfs":{"total":289910292480,"available":287659151360,"internally_reserved":0,"allocated":1373364224,"data_stored":1372509211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":877710342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,7,10,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8101864,"num_objects":-1977,"num_object_clones":0,"num_object_copies":-3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":-1977,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6539,"num_write_kb":9121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004766"},"pg_stats":[{"pgid":"2.f","version":"45'10750","reported_seq":10825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.947483+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:42.947483+0000","last_peered":"2024-09-05T13:16:42.947483+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:42.947483+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41869312,"num_objects":10222,"num_object_clones":0,"num_object_copies":20444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10750,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5801},{"shards":"0,2","objects":4156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10639","reported_seq":10743,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.880724+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:42.880724+0000","last_peered":"2024-09-05T13:16:42.880724+0000","last_clean":"2024-09-05T13:16:42.880724+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:42.880724+0000","last_undegraded":"2024-09-05T13:16:42.880724+0000","last_fullsized":"2024-09-05T13:16:42.880724+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431040,"num_objects":10115,"num_object_clones":0,"num_object_copies":20230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8649,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10882","reported_seq":11010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.857019+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:42.857019+0000","last_peered":"2024-09-05T13:16:42.857019+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:42.857019+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42242072,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8693,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4210},{"shards":"2","objects":5822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10796","reported_seq":10865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.906339+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:42.906339+0000","last_peered":"2024-09-05T13:16:42.906339+0000","last_clean":"2024-09-05T13:16:42.906339+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:42.906339+0000","last_undegraded":"2024-09-05T13:16:42.906339+0000","last_fullsized":"2024-09-05T13:16:42.906339+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41902080,"num_objects":10230,"num_object_clones":0,"num_object_copies":20460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10796,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10728","reported_seq":10862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.912915+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:42.912915+0000","last_peered":"2024-09-05T13:16:42.912915+0000","last_clean":"2024-09-05T13:16:42.912915+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:42.912915+0000","last_undegraded":"2024-09-05T13:16:42.912915+0000","last_fullsized":"2024-09-05T13:16:42.912915+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41594880,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10841","reported_seq":10979,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.949974+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:42.949974+0000","last_peered":"2024-09-05T13:16:42.949974+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:42.949974+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42246144,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5863},{"shards":"0,2","objects":4190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10575","reported_seq":11881,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.981953+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:42.981953+0000","last_peered":"2024-09-05T13:16:42.981953+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:42.981953+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41177088,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":2392064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6201},{"shards":"0,2","objects":3851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10820","reported_seq":10950,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:43.002879+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:43.002879+0000","last_peered":"2024-09-05T13:16:43.002879+0000","last_clean":"2024-09-05T13:16:43.002879+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:43.002879+0000","last_undegraded":"2024-09-05T13:16:43.002879+0000","last_fullsized":"2024-09-05T13:16:43.002879+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42123264,"num_objects":10284,"num_object_clones":0,"num_object_copies":20568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10820","reported_seq":10955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.786235+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:42.786235+0000","last_peered":"2024-09-05T13:16:42.786235+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:42.786235+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42078208,"num_objects":10273,"num_object_clones":0,"num_object_copies":20546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8474,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4181},{"shards":"2","objects":5822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10701","reported_seq":10839,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.942010+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:42.942010+0000","last_peered":"2024-09-05T13:16:42.942010+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:42.942010+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41816064,"num_objects":10209,"num_object_clones":0,"num_object_copies":20418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7068,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":4212}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10736","reported_seq":19635,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.981814+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:42.981814+0000","last_peered":"2024-09-05T13:16:42.981814+0000","last_clean":"2024-09-05T13:16:42.981814+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:42.981814+0000","last_undegraded":"2024-09-05T13:16:42.981814+0000","last_fullsized":"2024-09-05T13:16:42.981814+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41676800,"num_objects":10175,"num_object_clones":0,"num_object_copies":20350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8331,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10556","reported_seq":10678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.973720+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:42.973720+0000","last_peered":"2024-09-05T13:16:42.973720+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:42.973720+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8065,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4125},{"shards":"2","objects":5639}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10701","reported_seq":10841,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.954834+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:42.954834+0000","last_peered":"2024-09-05T13:16:42.954834+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:42.954834+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41684992,"num_objects":10177,"num_object_clones":0,"num_object_copies":20354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6852,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5758},{"shards":"0,2","objects":4158}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10824","reported_seq":11011,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.933080+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:42.933080+0000","last_peered":"2024-09-05T13:16:42.933080+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:42.933080+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42106880,"num_objects":10280,"num_object_clones":0,"num_object_copies":20560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":4176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10800","reported_seq":10939,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.933806+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:42.933806+0000","last_peered":"2024-09-05T13:16:42.933806+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:42.933806+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42106880,"num_objects":10280,"num_object_clones":0,"num_object_copies":20560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":4308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10864","reported_seq":11004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.959590+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:42.959590+0000","last_peered":"2024-09-05T13:16:42.959590+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:42.959590+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42303971,"num_objects":10329,"num_object_clones":0,"num_object_copies":20658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7532,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5764},{"shards":"0,2","objects":4303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669426171,"num_objects":163436,"num_object_clones":0,"num_object_copies":326872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":163436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127751,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4976,"num_bytes_recovered":20373504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1370390528,"data_stored":1370299285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754496,"kb_used_data":470128,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93617344,"statfs":{"total":96636764160,"available":95864160256,"internally_reserved":0,"allocated":481411072,"data_stored":481151515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":909776,"kb_used_data":563008,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93462064,"statfs":{"total":96636764160,"available":95705153536,"internally_reserved":0,"allocated":576520192,"data_stored":576260630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534108,"kb_used_data":308040,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93837732,"statfs":{"total":96636764160,"available":96089837568,"internally_reserved":0,"allocated":315432960,"data_stored":315097066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":231468719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":574885888,"data_stored":574882275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314937344,"data_stored":314857399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480567296,"data_stored":480559611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:44.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:44.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10750", 'reported_seq': 10825, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.947483+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:42.947483+0000', 'last_peered': '2024-09-05T13:16:42.947483+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:42.947483+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41869312, 'num_objects': 10222, 'num_object_clones': 0, 'num_object_copies': 20444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10750, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5801}, {'shards': '0,2', 'objects': 4156}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:44.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:44.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10882", 'reported_seq': 11010, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.857019+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:42.857019+0000', 'last_peered': '2024-09-05T13:16:42.857019+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:42.857019+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42242072, 'num_objects': 10314, 'num_object_clones': 0, 'num_object_copies': 20628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10314, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8693, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4210}, {'shards': '2', 'objects': 5822}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:44.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:44.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10841", 'reported_seq': 10979, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.949974+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:42.949974+0000', 'last_peered': '2024-09-05T13:16:42.949974+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:42.949974+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42246144, 'num_objects': 10314, 'num_object_clones': 0, 'num_object_copies': 20628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10314, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7296, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5863}, {'shards': '0,2', 'objects': 4190}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10575", 'reported_seq': 11881, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.981953+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:42.981953+0000', 'last_peered': '2024-09-05T13:16:42.981953+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:42.981953+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41177088, 'num_objects': 10053, 'num_object_clones': 0, 'num_object_copies': 20106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 3851, 'num_objects_unfound': 0, 'num_objects_dirty': 10053, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6894, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 584, 'num_bytes_recovered': 2392064, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6201}, {'shards': '0,2', 'objects': 3851}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10820", 'reported_seq': 10955, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.786235+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:42.786235+0000', 'last_peered': '2024-09-05T13:16:42.786235+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:42.786235+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42078208, 'num_objects': 10273, 'num_object_clones': 0, 'num_object_copies': 20546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8474, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4181}, {'shards': '2', 'objects': 5822}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10701", 'reported_seq': 10839, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.942010+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:42.942010+0000', 'last_peered': '2024-09-05T13:16:42.942010+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:42.942010+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41816064, 'num_objects': 10209, 'num_object_clones': 0, 'num_object_copies': 20418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7068, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5755}, {'shards': '0,2', 'objects': 4212}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10556", 'reported_seq': 10678, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.973720+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:42.973720+0000', 'last_peered': '2024-09-05T13:16:42.973720+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:42.973720+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41066496, 'num_objects': 10026, 'num_object_clones': 0, 'num_object_copies': 20052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8065, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4125}, {'shards': '2', 'objects': 5639}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10701", 'reported_seq': 10841, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.954834+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:42.954834+0000', 'last_peered': '2024-09-05T13:16:42.954834+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:42.954834+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41684992, 'num_objects': 10177, 'num_object_clones': 0, 'num_object_copies': 20354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6852, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5758}, {'shards': '0,2', 'objects': 4158}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:44.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10824", 'reported_seq': 11011, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.933080+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:42.933080+0000', 'last_peered': '2024-09-05T13:16:42.933080+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:42.933080+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42106880, 'num_objects': 10280, 'num_object_clones': 0, 'num_object_copies': 20560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6723, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5836}, {'shards': '0,2', 'objects': 4176}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:44.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:44.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10800", 'reported_seq': 10939, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.933806+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:42.933806+0000', 'last_peered': '2024-09-05T13:16:42.933806+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:42.933806+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42106880, 'num_objects': 10280, 'num_object_clones': 0, 'num_object_copies': 20560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6930, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5714}, {'shards': '0,2', 'objects': 4308}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:44.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:44.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10864", 'reported_seq': 11004, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.959590+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:42.959590+0000', 'last_peered': '2024-09-05T13:16:42.959590+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:42.959590+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42303971, 'num_objects': 10329, 'num_object_clones': 0, 'num_object_copies': 20658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7532, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5764}, {'shards': '0,2', 'objects': 4303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:44.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:44.756 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:44.756 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/326876 objects degraded (13.665%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":331,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":163438,"data_bytes":669426190,"bytes_used":2251141120,"bytes_avail":287659151360,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":326876,"degraded_ratio":0.13665426644966286,"misplaced_objects":3851,"misplaced_total":326876,"misplaced_ratio":0.011781225908295501,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":87346,"recovering_keys_per_sec":0,"num_objects_recovered":256,"num_bytes_recovered":1048576,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":778016,"read_op_per_sec":0,"write_op_per_sec":544},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:44.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:44.781 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:45.381 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-05T13:16:43.604727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669426190,"num_objects":163438,"num_object_clones":0,"num_object_copies":326876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":163438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127753,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4977,"num_bytes_recovered":20373523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2198380,"kb_used_data":1341176,"kb_used_omap":12,"kb_used_meta":857139,"kb_avail":280917140,"statfs":{"total":289910292480,"available":287659151360,"internally_reserved":0,"allocated":1373364224,"data_stored":1372509211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":877710342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,7,10,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8101864,"num_objects":-1977,"num_object_clones":0,"num_object_copies":-3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-256,"num_objects_unfound":0,"num_objects_dirty":-1977,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6539,"num_write_kb":9121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004766"},"pg_stats":[{"pgid":"2.f","version":"45'10750","reported_seq":10825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.947483+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:42.947483+0000","last_peered":"2024-09-05T13:16:42.947483+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:42.947483+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41869312,"num_objects":10222,"num_object_clones":0,"num_object_copies":20444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10750,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5801},{"shards":"0,2","objects":4156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10639","reported_seq":10743,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.880724+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:42.880724+0000","last_peered":"2024-09-05T13:16:42.880724+0000","last_clean":"2024-09-05T13:16:42.880724+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:42.880724+0000","last_undegraded":"2024-09-05T13:16:42.880724+0000","last_fullsized":"2024-09-05T13:16:42.880724+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431040,"num_objects":10115,"num_object_clones":0,"num_object_copies":20230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8649,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10882","reported_seq":11010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.857019+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:42.857019+0000","last_peered":"2024-09-05T13:16:42.857019+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:42.857019+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42242072,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8693,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4210},{"shards":"2","objects":5822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10796","reported_seq":10865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.906339+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:42.906339+0000","last_peered":"2024-09-05T13:16:42.906339+0000","last_clean":"2024-09-05T13:16:42.906339+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:42.906339+0000","last_undegraded":"2024-09-05T13:16:42.906339+0000","last_fullsized":"2024-09-05T13:16:42.906339+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41902080,"num_objects":10230,"num_object_clones":0,"num_object_copies":20460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10796,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10728","reported_seq":10862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.912915+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:42.912915+0000","last_peered":"2024-09-05T13:16:42.912915+0000","last_clean":"2024-09-05T13:16:42.912915+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:42.912915+0000","last_undegraded":"2024-09-05T13:16:42.912915+0000","last_fullsized":"2024-09-05T13:16:42.912915+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41594880,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10841","reported_seq":10979,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.949974+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:42.949974+0000","last_peered":"2024-09-05T13:16:42.949974+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:42.949974+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42246144,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5863},{"shards":"0,2","objects":4190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10575","reported_seq":11881,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.981953+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:42.981953+0000","last_peered":"2024-09-05T13:16:42.981953+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:42.981953+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41177088,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":2392064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6201},{"shards":"0,2","objects":3851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10820","reported_seq":10950,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:43.002879+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:43.002879+0000","last_peered":"2024-09-05T13:16:43.002879+0000","last_clean":"2024-09-05T13:16:43.002879+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:43.002879+0000","last_undegraded":"2024-09-05T13:16:43.002879+0000","last_fullsized":"2024-09-05T13:16:43.002879+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42123264,"num_objects":10284,"num_object_clones":0,"num_object_copies":20568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10820","reported_seq":10955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.786235+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:42.786235+0000","last_peered":"2024-09-05T13:16:42.786235+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:42.786235+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42078208,"num_objects":10273,"num_object_clones":0,"num_object_copies":20546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8474,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4181},{"shards":"2","objects":5822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10701","reported_seq":10839,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.942010+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:42.942010+0000","last_peered":"2024-09-05T13:16:42.942010+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:42.942010+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41816064,"num_objects":10209,"num_object_clones":0,"num_object_copies":20418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7068,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":4212}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10736","reported_seq":19635,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.981814+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:42.981814+0000","last_peered":"2024-09-05T13:16:42.981814+0000","last_clean":"2024-09-05T13:16:42.981814+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:42.981814+0000","last_undegraded":"2024-09-05T13:16:42.981814+0000","last_fullsized":"2024-09-05T13:16:42.981814+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41676800,"num_objects":10175,"num_object_clones":0,"num_object_copies":20350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8331,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10556","reported_seq":10678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.973720+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:42.973720+0000","last_peered":"2024-09-05T13:16:42.973720+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:42.973720+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8065,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4125},{"shards":"2","objects":5639}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10701","reported_seq":10841,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.954834+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:42.954834+0000","last_peered":"2024-09-05T13:16:42.954834+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:42.954834+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41684992,"num_objects":10177,"num_object_clones":0,"num_object_copies":20354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6852,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5758},{"shards":"0,2","objects":4158}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10824","reported_seq":11011,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.933080+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:42.933080+0000","last_peered":"2024-09-05T13:16:42.933080+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:42.933080+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42106880,"num_objects":10280,"num_object_clones":0,"num_object_copies":20560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":4176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10800","reported_seq":10939,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.933806+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:42.933806+0000","last_peered":"2024-09-05T13:16:42.933806+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:42.933806+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42106880,"num_objects":10280,"num_object_clones":0,"num_object_copies":20560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":4308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10864","reported_seq":11004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.959590+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:42.959590+0000","last_peered":"2024-09-05T13:16:42.959590+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:42.959590+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42303971,"num_objects":10329,"num_object_clones":0,"num_object_copies":20658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7532,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5764},{"shards":"0,2","objects":4303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669426171,"num_objects":163436,"num_object_clones":0,"num_object_copies":326872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":163436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127751,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4976,"num_bytes_recovered":20373504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1370390528,"data_stored":1370299285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754496,"kb_used_data":470128,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93617344,"statfs":{"total":96636764160,"available":95864160256,"internally_reserved":0,"allocated":481411072,"data_stored":481151515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":909776,"kb_used_data":563008,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93462064,"statfs":{"total":96636764160,"available":95705153536,"internally_reserved":0,"allocated":576520192,"data_stored":576260630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534108,"kb_used_data":308040,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93837732,"statfs":{"total":96636764160,"available":96089837568,"internally_reserved":0,"allocated":315432960,"data_stored":315097066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":231468719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":574885888,"data_stored":574882275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314937344,"data_stored":314857399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480567296,"data_stored":480559611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:48.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:48.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:48.948 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-05T13:16:47.606428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669426190,"num_objects":163438,"num_object_clones":0,"num_object_copies":326876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":163438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127753,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4977,"num_bytes_recovered":20373523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2223968,"kb_used_data":1340400,"kb_used_omap":12,"kb_used_meta":883507,"kb_avail":280891552,"statfs":{"total":289910292480,"available":287632949248,"internally_reserved":0,"allocated":1372569600,"data_stored":1371714587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":904711174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,7,7,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16953320,"num_objects":-4138,"num_object_clones":0,"num_object_copies":-8276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":-4138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4378,"num_write_kb":477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004882"},"pg_stats":[{"pgid":"2.f","version":"45'10750","reported_seq":10825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.947483+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:42.947483+0000","last_peered":"2024-09-05T13:16:42.947483+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:42.947483+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41869312,"num_objects":10222,"num_object_clones":0,"num_object_copies":20444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10750,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5801},{"shards":"0,2","objects":4156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10639","reported_seq":10743,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.880724+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:42.880724+0000","last_peered":"2024-09-05T13:16:42.880724+0000","last_clean":"2024-09-05T13:16:42.880724+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:42.880724+0000","last_undegraded":"2024-09-05T13:16:42.880724+0000","last_fullsized":"2024-09-05T13:16:42.880724+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431040,"num_objects":10115,"num_object_clones":0,"num_object_copies":20230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8649,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10882","reported_seq":11010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.857019+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:42.857019+0000","last_peered":"2024-09-05T13:16:42.857019+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:42.857019+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42242072,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8693,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4210},{"shards":"2","objects":5822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10796","reported_seq":10865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.906339+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:42.906339+0000","last_peered":"2024-09-05T13:16:42.906339+0000","last_clean":"2024-09-05T13:16:42.906339+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:42.906339+0000","last_undegraded":"2024-09-05T13:16:42.906339+0000","last_fullsized":"2024-09-05T13:16:42.906339+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41902080,"num_objects":10230,"num_object_clones":0,"num_object_copies":20460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10796,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10728","reported_seq":10862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.912915+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:42.912915+0000","last_peered":"2024-09-05T13:16:42.912915+0000","last_clean":"2024-09-05T13:16:42.912915+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:42.912915+0000","last_undegraded":"2024-09-05T13:16:42.912915+0000","last_fullsized":"2024-09-05T13:16:42.912915+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41594880,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10841","reported_seq":10979,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.949974+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:42.949974+0000","last_peered":"2024-09-05T13:16:42.949974+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:42.949974+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42246144,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5863},{"shards":"0,2","objects":4190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10575","reported_seq":11881,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.981953+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:42.981953+0000","last_peered":"2024-09-05T13:16:42.981953+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:42.981953+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41177088,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":2392064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6201},{"shards":"0,2","objects":3851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10820","reported_seq":10950,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:43.002879+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:43.002879+0000","last_peered":"2024-09-05T13:16:43.002879+0000","last_clean":"2024-09-05T13:16:43.002879+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:43.002879+0000","last_undegraded":"2024-09-05T13:16:43.002879+0000","last_fullsized":"2024-09-05T13:16:43.002879+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42123264,"num_objects":10284,"num_object_clones":0,"num_object_copies":20568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10820","reported_seq":10955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.786235+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:42.786235+0000","last_peered":"2024-09-05T13:16:42.786235+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:42.786235+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42078208,"num_objects":10273,"num_object_clones":0,"num_object_copies":20546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8474,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4181},{"shards":"2","objects":5822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10701","reported_seq":10839,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.942010+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:42.942010+0000","last_peered":"2024-09-05T13:16:42.942010+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:42.942010+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41816064,"num_objects":10209,"num_object_clones":0,"num_object_copies":20418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7068,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":4212}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10736","reported_seq":19635,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.981814+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:42.981814+0000","last_peered":"2024-09-05T13:16:42.981814+0000","last_clean":"2024-09-05T13:16:42.981814+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:42.981814+0000","last_undegraded":"2024-09-05T13:16:42.981814+0000","last_fullsized":"2024-09-05T13:16:42.981814+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41676800,"num_objects":10175,"num_object_clones":0,"num_object_copies":20350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8331,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10556","reported_seq":10678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.973720+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:42.973720+0000","last_peered":"2024-09-05T13:16:42.973720+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:42.973720+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8065,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4125},{"shards":"2","objects":5639}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10701","reported_seq":10841,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.954834+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:42.954834+0000","last_peered":"2024-09-05T13:16:42.954834+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:42.954834+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41684992,"num_objects":10177,"num_object_clones":0,"num_object_copies":20354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6852,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5758},{"shards":"0,2","objects":4158}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10824","reported_seq":11011,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.933080+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:42.933080+0000","last_peered":"2024-09-05T13:16:42.933080+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:42.933080+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42106880,"num_objects":10280,"num_object_clones":0,"num_object_copies":20560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":4176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10800","reported_seq":10939,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.933806+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:42.933806+0000","last_peered":"2024-09-05T13:16:42.933806+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:42.933806+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42106880,"num_objects":10280,"num_object_clones":0,"num_object_copies":20560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":4308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10864","reported_seq":11004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.959590+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:42.959590+0000","last_peered":"2024-09-05T13:16:42.959590+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:42.959590+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42303971,"num_objects":10329,"num_object_clones":0,"num_object_copies":20658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7532,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5764},{"shards":"0,2","objects":4303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669426171,"num_objects":163436,"num_object_clones":0,"num_object_copies":326872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":163436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127751,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4976,"num_bytes_recovered":20373504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1369587712,"data_stored":1369496469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754496,"kb_used_data":470128,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93617344,"statfs":{"total":96636764160,"available":95864160256,"internally_reserved":0,"allocated":481411072,"data_stored":481151515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":909776,"kb_used_data":563008,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93462064,"statfs":{"total":96636764160,"available":95705153536,"internally_reserved":0,"allocated":576520192,"data_stored":576260630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559696,"kb_used_data":307264,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812144,"statfs":{"total":96636764160,"available":96063635456,"internally_reserved":0,"allocated":314638336,"data_stored":314302442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":574885888,"data_stored":574882275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314134528,"data_stored":314054583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480567296,"data_stored":480559611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:48.949 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:49.528 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:49.553 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-05T13:16:47.606428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669426190,"num_objects":163438,"num_object_clones":0,"num_object_copies":326876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":163438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127753,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4977,"num_bytes_recovered":20373523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2223968,"kb_used_data":1340400,"kb_used_omap":12,"kb_used_meta":883507,"kb_avail":280891552,"statfs":{"total":289910292480,"available":287632949248,"internally_reserved":0,"allocated":1372569600,"data_stored":1371714587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":904711174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,7,7,7,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16953320,"num_objects":-4138,"num_object_clones":0,"num_object_copies":-8276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":-4138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4378,"num_write_kb":477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004882"},"pg_stats":[{"pgid":"2.f","version":"45'10750","reported_seq":10825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.947483+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:42.947483+0000","last_peered":"2024-09-05T13:16:42.947483+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:42.947483+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41869312,"num_objects":10222,"num_object_clones":0,"num_object_copies":20444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10750,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5801},{"shards":"0,2","objects":4156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10639","reported_seq":10743,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.880724+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:42.880724+0000","last_peered":"2024-09-05T13:16:42.880724+0000","last_clean":"2024-09-05T13:16:42.880724+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:42.880724+0000","last_undegraded":"2024-09-05T13:16:42.880724+0000","last_fullsized":"2024-09-05T13:16:42.880724+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431040,"num_objects":10115,"num_object_clones":0,"num_object_copies":20230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8649,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10882","reported_seq":11010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.857019+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:42.857019+0000","last_peered":"2024-09-05T13:16:42.857019+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:42.857019+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42242072,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8693,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4210},{"shards":"2","objects":5822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10796","reported_seq":10865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.906339+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:42.906339+0000","last_peered":"2024-09-05T13:16:42.906339+0000","last_clean":"2024-09-05T13:16:42.906339+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:42.906339+0000","last_undegraded":"2024-09-05T13:16:42.906339+0000","last_fullsized":"2024-09-05T13:16:42.906339+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41902080,"num_objects":10230,"num_object_clones":0,"num_object_copies":20460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10796,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10728","reported_seq":10862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.912915+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:42.912915+0000","last_peered":"2024-09-05T13:16:42.912915+0000","last_clean":"2024-09-05T13:16:42.912915+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:42.912915+0000","last_undegraded":"2024-09-05T13:16:42.912915+0000","last_fullsized":"2024-09-05T13:16:42.912915+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41594880,"num_objects":10155,"num_object_clones":0,"num_object_copies":20310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10841","reported_seq":10979,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.949974+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:42.949974+0000","last_peered":"2024-09-05T13:16:42.949974+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:42.949974+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42246144,"num_objects":10314,"num_object_clones":0,"num_object_copies":20628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5863},{"shards":"0,2","objects":4190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10575","reported_seq":11881,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.981953+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:42.981953+0000","last_peered":"2024-09-05T13:16:42.981953+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:42.981953+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41177088,"num_objects":10053,"num_object_clones":0,"num_object_copies":20106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":10053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":2392064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6201},{"shards":"0,2","objects":3851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10820","reported_seq":10950,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:43.002879+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:43.002879+0000","last_peered":"2024-09-05T13:16:43.002879+0000","last_clean":"2024-09-05T13:16:43.002879+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:43.002879+0000","last_undegraded":"2024-09-05T13:16:43.002879+0000","last_fullsized":"2024-09-05T13:16:43.002879+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42123264,"num_objects":10284,"num_object_clones":0,"num_object_copies":20568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7430,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10820","reported_seq":10955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.786235+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:42.786235+0000","last_peered":"2024-09-05T13:16:42.786235+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:42.786235+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42078208,"num_objects":10273,"num_object_clones":0,"num_object_copies":20546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8474,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4181},{"shards":"2","objects":5822}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10701","reported_seq":10839,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.942010+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:42.942010+0000","last_peered":"2024-09-05T13:16:42.942010+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:42.942010+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41816064,"num_objects":10209,"num_object_clones":0,"num_object_copies":20418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7068,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":4212}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10736","reported_seq":19635,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:42.981814+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:42.981814+0000","last_peered":"2024-09-05T13:16:42.981814+0000","last_clean":"2024-09-05T13:16:42.981814+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:42.981814+0000","last_undegraded":"2024-09-05T13:16:42.981814+0000","last_fullsized":"2024-09-05T13:16:42.981814+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41676800,"num_objects":10175,"num_object_clones":0,"num_object_copies":20350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8331,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10556","reported_seq":10678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.973720+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:42.973720+0000","last_peered":"2024-09-05T13:16:42.973720+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:42.973720+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8065,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4125},{"shards":"2","objects":5639}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10701","reported_seq":10841,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.954834+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:42.954834+0000","last_peered":"2024-09-05T13:16:42.954834+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:42.954834+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41684992,"num_objects":10177,"num_object_clones":0,"num_object_copies":20354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6852,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5758},{"shards":"0,2","objects":4158}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10824","reported_seq":11011,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.933080+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:42.933080+0000","last_peered":"2024-09-05T13:16:42.933080+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:42.933080+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42106880,"num_objects":10280,"num_object_clones":0,"num_object_copies":20560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":4176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10800","reported_seq":10939,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.933806+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:42.933806+0000","last_peered":"2024-09-05T13:16:42.933806+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:42.933806+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42106880,"num_objects":10280,"num_object_clones":0,"num_object_copies":20560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":4308}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10864","reported_seq":11004,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:42.959590+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:42.959590+0000","last_peered":"2024-09-05T13:16:42.959590+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:42.959590+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42303971,"num_objects":10329,"num_object_clones":0,"num_object_copies":20658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7532,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5764},{"shards":"0,2","objects":4303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669426171,"num_objects":163436,"num_object_clones":0,"num_object_copies":326872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3851,"num_objects_unfound":0,"num_objects_dirty":163436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127751,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4976,"num_bytes_recovered":20373504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1369587712,"data_stored":1369496469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754496,"kb_used_data":470128,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93617344,"statfs":{"total":96636764160,"available":95864160256,"internally_reserved":0,"allocated":481411072,"data_stored":481151515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":909776,"kb_used_data":563008,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93462064,"statfs":{"total":96636764160,"available":95705153536,"internally_reserved":0,"allocated":576520192,"data_stored":576260630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559696,"kb_used_data":307264,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812144,"statfs":{"total":96636764160,"available":96063635456,"internally_reserved":0,"allocated":314638336,"data_stored":314302442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":574885888,"data_stored":574882275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314134528,"data_stored":314054583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":480567296,"data_stored":480559611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:49.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:49.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10750", 'reported_seq': 10825, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.947483+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:42.947483+0000', 'last_peered': '2024-09-05T13:16:42.947483+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:42.947483+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41869312, 'num_objects': 10222, 'num_object_clones': 0, 'num_object_copies': 20444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10750, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5801}, {'shards': '0,2', 'objects': 4156}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:49.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:49.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10882", 'reported_seq': 11010, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.857019+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:42.857019+0000', 'last_peered': '2024-09-05T13:16:42.857019+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:42.857019+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42242072, 'num_objects': 10314, 'num_object_clones': 0, 'num_object_copies': 20628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10314, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8693, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4210}, {'shards': '2', 'objects': 5822}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10841", 'reported_seq': 10979, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.949974+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:42.949974+0000', 'last_peered': '2024-09-05T13:16:42.949974+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:42.949974+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42246144, 'num_objects': 10314, 'num_object_clones': 0, 'num_object_copies': 20628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10314, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7296, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5863}, {'shards': '0,2', 'objects': 4190}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10575", 'reported_seq': 11881, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.981953+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:42.981953+0000', 'last_peered': '2024-09-05T13:16:42.981953+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:42.981953+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41177088, 'num_objects': 10053, 'num_object_clones': 0, 'num_object_copies': 20106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 3851, 'num_objects_unfound': 0, 'num_objects_dirty': 10053, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6894, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 584, 'num_bytes_recovered': 2392064, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6201}, {'shards': '0,2', 'objects': 3851}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10820", 'reported_seq': 10955, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.786235+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:42.786235+0000', 'last_peered': '2024-09-05T13:16:42.786235+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:42.786235+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42078208, 'num_objects': 10273, 'num_object_clones': 0, 'num_object_copies': 20546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8474, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4181}, {'shards': '2', 'objects': 5822}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10701", 'reported_seq': 10839, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.942010+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:42.942010+0000', 'last_peered': '2024-09-05T13:16:42.942010+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:42.942010+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41816064, 'num_objects': 10209, 'num_object_clones': 0, 'num_object_copies': 20418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7068, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5755}, {'shards': '0,2', 'objects': 4212}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10556", 'reported_seq': 10678, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.973720+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:42.973720+0000', 'last_peered': '2024-09-05T13:16:42.973720+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:42.973720+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41066496, 'num_objects': 10026, 'num_object_clones': 0, 'num_object_copies': 20052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8065, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4125}, {'shards': '2', 'objects': 5639}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:49.555 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:49.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10701", 'reported_seq': 10841, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.954834+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:42.954834+0000', 'last_peered': '2024-09-05T13:16:42.954834+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:42.954834+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41684992, 'num_objects': 10177, 'num_object_clones': 0, 'num_object_copies': 20354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6852, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5758}, {'shards': '0,2', 'objects': 4158}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:49.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:49.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10824", 'reported_seq': 11011, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.933080+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:42.933080+0000', 'last_peered': '2024-09-05T13:16:42.933080+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:42.933080+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42106880, 'num_objects': 10280, 'num_object_clones': 0, 'num_object_copies': 20560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6723, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5836}, {'shards': '0,2', 'objects': 4176}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:49.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:49.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10800", 'reported_seq': 10939, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.933806+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:42.933806+0000', 'last_peered': '2024-09-05T13:16:42.933806+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:42.933806+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42106880, 'num_objects': 10280, 'num_object_clones': 0, 'num_object_copies': 20560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6930, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5714}, {'shards': '0,2', 'objects': 4308}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:49.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:49.556 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10864", 'reported_seq': 11004, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:42.959590+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:42.959590+0000', 'last_peered': '2024-09-05T13:16:42.959590+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:42.959590+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42303971, 'num_objects': 10329, 'num_object_clones': 0, 'num_object_copies': 20658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7532, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5764}, {'shards': '0,2', 'objects': 4303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:49.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:50.235 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:50.235 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/322528 objects degraded (13.850%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":337,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":161264,"data_bytes":660521486,"bytes_used":2257891328,"bytes_avail":287652401152,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":322528,"degraded_ratio":0.13849650262922908,"misplaced_objects":3762,"misplaced_total":322528,"misplaced_ratio":0.011664103581704534,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":89392,"recovering_keys_per_sec":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":40687,"read_op_per_sec":0,"write_op_per_sec":545},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:50.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:50.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:50.810 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:50.813 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:50.841 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-05T13:16:49.607240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660521486,"num_objects":161264,"num_object_clones":0,"num_object_copies":322528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3762,"num_objects_unfound":0,"num_objects_dirty":161264,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129927,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5066,"num_bytes_recovered":20738067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2204972,"kb_used_data":1329720,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":280910548,"statfs":{"total":289910292480,"available":287652401152,"internally_reserved":0,"allocated":1361633280,"data_stored":1360778267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":896191494},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,8,16,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25858024,"num_objects":-6312,"num_object_clones":0,"num_object_copies":-12624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-262,"num_objects_unfound":0,"num_objects_dirty":-6312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6552,"num_write_kb":477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004916"},"pg_stats":[{"pgid":"2.f","version":"45'10894","reported_seq":10969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:47.983441+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:47.983441+0000","last_peered":"2024-09-05T13:16:47.983441+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:47.983441+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41279488,"num_objects":10078,"num_object_clones":0,"num_object_copies":20156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10894,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5657},{"shards":"0,2","objects":4012}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10784","reported_seq":10887,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:48.039335+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:48.039335+0000","last_peered":"2024-09-05T13:16:48.039335+0000","last_clean":"2024-09-05T13:16:48.039335+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:48.039335+0000","last_undegraded":"2024-09-05T13:16:48.039335+0000","last_fullsized":"2024-09-05T13:16:48.039335+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40837120,"num_objects":9970,"num_object_clones":0,"num_object_copies":19940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8794,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11010","reported_seq":11138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:47.946935+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:47.946935+0000","last_peered":"2024-09-05T13:16:47.946935+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:47.946935+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41717784,"num_objects":10186,"num_object_clones":0,"num_object_copies":20372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10186,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8821,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4082},{"shards":"2","objects":5694}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10924","reported_seq":10994,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:48.021112+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:48.021112+0000","last_peered":"2024-09-05T13:16:48.021112+0000","last_clean":"2024-09-05T13:16:48.021112+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:48.021112+0000","last_undegraded":"2024-09-05T13:16:48.021112+0000","last_fullsized":"2024-09-05T13:16:48.021112+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41377792,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10924,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10861","reported_seq":10995,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:48.016921+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:48.016921+0000","last_peered":"2024-09-05T13:16:48.016921+0000","last_clean":"2024-09-05T13:16:48.016921+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:48.016921+0000","last_undegraded":"2024-09-05T13:16:48.016921+0000","last_fullsized":"2024-09-05T13:16:48.016921+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41050112,"num_objects":10022,"num_object_clones":0,"num_object_copies":20044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7401,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11011","reported_seq":11147,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:47.931008+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:47.931008+0000","last_peered":"2024-09-05T13:16:47.931008+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:47.931008+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":10144,"num_object_clones":0,"num_object_copies":20288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7466,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5693},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10699","reported_seq":12185,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:47.975281+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:47.975281+0000","last_peered":"2024-09-05T13:16:47.975281+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:47.975281+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669184,"num_objects":9929,"num_object_clones":0,"num_object_copies":19858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3762,"num_objects_unfound":0,"num_objects_dirty":9929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7018,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2756608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6166},{"shards":"0,2","objects":3762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10943","reported_seq":11075,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:47.948989+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:47.948989+0000","last_peered":"2024-09-05T13:16:47.948989+0000","last_clean":"2024-09-05T13:16:47.948989+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:47.948989+0000","last_undegraded":"2024-09-05T13:16:47.948989+0000","last_fullsized":"2024-09-05T13:16:47.948989+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41619456,"num_objects":10161,"num_object_clones":0,"num_object_copies":20322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7553,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10950","reported_seq":11085,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:48.002543+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:48.002543+0000","last_peered":"2024-09-05T13:16:48.002543+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:48.002543+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41545728,"num_objects":10143,"num_object_clones":0,"num_object_copies":20286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8604,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4051},{"shards":"2","objects":5692}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10833","reported_seq":10970,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:47.962666+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:47.962666+0000","last_peered":"2024-09-05T13:16:47.962666+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:47.962666+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41275392,"num_objects":10077,"num_object_clones":0,"num_object_copies":20154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7200,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5623},{"shards":"0,2","objects":4080}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10870","reported_seq":19769,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:47.934426+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:47.934426+0000","last_peered":"2024-09-05T13:16:47.934426+0000","last_clean":"2024-09-05T13:16:47.934426+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:47.934426+0000","last_undegraded":"2024-09-05T13:16:47.934426+0000","last_fullsized":"2024-09-05T13:16:47.934426+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41127936,"num_objects":10041,"num_object_clones":0,"num_object_copies":20082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8465,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10679","reported_seq":10802,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:47.971720+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:47.971720+0000","last_peered":"2024-09-05T13:16:47.971720+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:47.971720+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40562688,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8188,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4002},{"shards":"2","objects":5516}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10839","reported_seq":10977,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:47.985601+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:47.985601+0000","last_peered":"2024-09-05T13:16:47.985601+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:47.985601+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41119744,"num_objects":10039,"num_object_clones":0,"num_object_copies":20078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6990,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5620},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10977","reported_seq":11164,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:48.007379+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:48.007379+0000","last_peered":"2024-09-05T13:16:48.007379+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:48.007379+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41480192,"num_objects":10127,"num_object_clones":0,"num_object_copies":20254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6876,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5683},{"shards":"0,2","objects":4023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10943","reported_seq":11082,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:48.026193+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:48.026193+0000","last_peered":"2024-09-05T13:16:48.026193+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:48.026193+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41521152,"num_objects":10137,"num_object_clones":0,"num_object_copies":20274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7073,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5571},{"shards":"0,2","objects":4165}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10990","reported_seq":11130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:47.964670+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:47.964670+0000","last_peered":"2024-09-05T13:16:47.964670+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:47.964670+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41787875,"num_objects":10203,"num_object_clones":0,"num_object_copies":20406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7658,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5638},{"shards":"0,2","objects":4177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":660521467,"num_objects":161262,"num_object_clones":0,"num_object_copies":322524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3762,"num_objects_unfound":0,"num_objects_dirty":161262,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129925,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5065,"num_bytes_recovered":20738048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1358512128,"data_stored":1358420885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754504,"kb_used_data":465972,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93617336,"statfs":{"total":96636764160,"available":95864152064,"internally_reserved":0,"allocated":477155328,"data_stored":476895771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":890772,"kb_used_data":556484,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93481068,"statfs":{"total":96636764160,"available":95724613632,"internally_reserved":0,"allocated":569839616,"data_stored":569580054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559696,"kb_used_data":307264,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93812144,"statfs":{"total":96636764160,"available":96063635456,"internally_reserved":0,"allocated":314638336,"data_stored":314302442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568102912,"data_stored":568099299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314134528,"data_stored":314054583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":476274688,"data_stored":476267003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:53.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:54.432 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:54.434 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:54.456 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-05T13:16:53.608607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651489806,"num_objects":159059,"num_object_clones":0,"num_object_copies":318118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":159059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132132,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5146,"num_bytes_recovered":21065747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2226716,"kb_used_data":1316772,"kb_used_omap":12,"kb_used_meta":909875,"kb_avail":280888804,"statfs":{"total":289910292480,"available":287630135296,"internally_reserved":0,"allocated":1348374528,"data_stored":1347519515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":931712006},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,9,7,8,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26963968,"num_objects":-6583,"num_object_clones":0,"num_object_copies":-13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":-6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6583,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004668"},"pg_stats":[{"pgid":"2.f","version":"45'10988","reported_seq":11063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:52.922851+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:52.922851+0000","last_peered":"2024-09-05T13:16:52.922851+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:52.922851+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10988,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5563},{"shards":"0,2","objects":3918}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10922","reported_seq":11027,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:52.951595+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:52.951595+0000","last_peered":"2024-09-05T13:16:52.951595+0000","last_clean":"2024-09-05T13:16:52.951595+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:52.951595+0000","last_undegraded":"2024-09-05T13:16:52.951595+0000","last_fullsized":"2024-09-05T13:16:52.951595+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40271872,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8932,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11156","reported_seq":11284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:52.951954+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:52.951954+0000","last_peered":"2024-09-05T13:16:52.951954+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:52.951954+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41119768,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8967,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3936},{"shards":"2","objects":5548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11046","reported_seq":11116,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.023882+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:53.023882+0000","last_peered":"2024-09-05T13:16:53.023882+0000","last_clean":"2024-09-05T13:16:53.023882+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:53.023882+0000","last_undegraded":"2024-09-05T13:16:53.023882+0000","last_fullsized":"2024-09-05T13:16:53.023882+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878080,"num_objects":9980,"num_object_clones":0,"num_object_copies":19960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11046,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11018","reported_seq":11152,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.125837+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:53.125837+0000","last_peered":"2024-09-05T13:16:53.125837+0000","last_clean":"2024-09-05T13:16:53.125837+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:53.125837+0000","last_undegraded":"2024-09-05T13:16:53.125837+0000","last_fullsized":"2024-09-05T13:16:53.125837+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7558,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11145","reported_seq":11283,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.080409+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:53.080409+0000","last_peered":"2024-09-05T13:16:53.080409+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:53.080409+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41000960,"num_objects":10010,"num_object_clones":0,"num_object_copies":20020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5559},{"shards":"0,2","objects":3886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10842","reported_seq":12487,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.142524+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:53.142524+0000","last_peered":"2024-09-05T13:16:53.142524+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:53.142524+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7161,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":753,"num_bytes_recovered":3084288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6103},{"shards":"0,2","objects":3682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11074","reported_seq":11206,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.038317+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:53.038317+0000","last_peered":"2024-09-05T13:16:53.038317+0000","last_clean":"2024-09-05T13:16:53.038317+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:53.038317+0000","last_undegraded":"2024-09-05T13:16:53.038317+0000","last_fullsized":"2024-09-05T13:16:53.038317+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41082880,"num_objects":10030,"num_object_clones":0,"num_object_copies":20060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11095","reported_seq":11230,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.031047+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:53.031047+0000","last_peered":"2024-09-05T13:16:53.031047+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:53.031047+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40951808,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8749,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3906},{"shards":"2","objects":5547}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10993","reported_seq":11131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.075114+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:53.075114+0000","last_peered":"2024-09-05T13:16:53.075114+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:53.075114+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620032,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7360,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3920}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11019","reported_seq":19920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:52.954371+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:52.954371+0000","last_peered":"2024-09-05T13:16:52.954371+0000","last_clean":"2024-09-05T13:16:52.954371+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:52.954371+0000","last_undegraded":"2024-09-05T13:16:52.954371+0000","last_fullsized":"2024-09-05T13:16:52.954371+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40517632,"num_objects":9892,"num_object_clones":0,"num_object_copies":19784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8614,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10806","reported_seq":10929,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.014337+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:53.014337+0000","last_peered":"2024-09-05T13:16:53.014337+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:53.014337+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40042496,"num_objects":9776,"num_object_clones":0,"num_object_copies":19552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8315,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3875},{"shards":"2","objects":5389}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10979","reported_seq":11119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.068788+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:53.068788+0000","last_peered":"2024-09-05T13:16:53.068788+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:53.068788+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546304,"num_objects":9899,"num_object_clones":0,"num_object_copies":19798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7130,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5480},{"shards":"0,2","objects":3880}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11111","reported_seq":11298,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.067090+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:53.067090+0000","last_peered":"2024-09-05T13:16:53.067090+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:53.067090+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40931328,"num_objects":9993,"num_object_clones":0,"num_object_copies":19986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7010,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5549},{"shards":"0,2","objects":3889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11098","reported_seq":11237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.113743+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:53.113743+0000","last_peered":"2024-09-05T13:16:53.113743+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:53.113743+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40886272,"num_objects":9982,"num_object_clones":0,"num_object_copies":19964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7228,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5416},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11120","reported_seq":11260,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.112776+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:53.112776+0000","last_peered":"2024-09-05T13:16:53.112776+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:53.112776+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41255395,"num_objects":10073,"num_object_clones":0,"num_object_copies":20146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7788,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5508},{"shards":"0,2","objects":4047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":651489787,"num_objects":159057,"num_object_clones":0,"num_object_copies":318114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":159057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132130,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5145,"num_bytes_recovered":21065728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1346494464,"data_stored":1346403221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754252,"kb_used_data":461564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93617588,"statfs":{"total":96636764160,"available":95864410112,"internally_reserved":0,"allocated":472641536,"data_stored":472381979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":891312,"kb_used_data":548696,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93480528,"statfs":{"total":96636764160,"available":95724060672,"internally_reserved":0,"allocated":561864704,"data_stored":561605142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,6,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581152,"kb_used_data":306512,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93790688,"statfs":{"total":96636764160,"available":96041664512,"internally_reserved":0,"allocated":313868288,"data_stored":313532394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561373184,"data_stored":561369571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313380864,"data_stored":313300919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471740416,"data_stored":471732731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:54.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:55.040 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:55.040 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:55.070 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-05T13:16:53.608607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651489806,"num_objects":159059,"num_object_clones":0,"num_object_copies":318118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":159059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132132,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5146,"num_bytes_recovered":21065747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2226716,"kb_used_data":1316772,"kb_used_omap":12,"kb_used_meta":909875,"kb_avail":280888804,"statfs":{"total":289910292480,"available":287630135296,"internally_reserved":0,"allocated":1348374528,"data_stored":1347519515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":931712006},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,9,7,8,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26963968,"num_objects":-6583,"num_object_clones":0,"num_object_copies":-13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":-6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6583,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004668"},"pg_stats":[{"pgid":"2.f","version":"45'10988","reported_seq":11063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:52.922851+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:52.922851+0000","last_peered":"2024-09-05T13:16:52.922851+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:52.922851+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10988,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5563},{"shards":"0,2","objects":3918}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10922","reported_seq":11027,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:52.951595+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:52.951595+0000","last_peered":"2024-09-05T13:16:52.951595+0000","last_clean":"2024-09-05T13:16:52.951595+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:52.951595+0000","last_undegraded":"2024-09-05T13:16:52.951595+0000","last_fullsized":"2024-09-05T13:16:52.951595+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40271872,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8932,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11156","reported_seq":11284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:52.951954+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:52.951954+0000","last_peered":"2024-09-05T13:16:52.951954+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:52.951954+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41119768,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8967,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3936},{"shards":"2","objects":5548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11046","reported_seq":11116,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.023882+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:53.023882+0000","last_peered":"2024-09-05T13:16:53.023882+0000","last_clean":"2024-09-05T13:16:53.023882+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:53.023882+0000","last_undegraded":"2024-09-05T13:16:53.023882+0000","last_fullsized":"2024-09-05T13:16:53.023882+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878080,"num_objects":9980,"num_object_clones":0,"num_object_copies":19960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11046,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11018","reported_seq":11152,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.125837+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:53.125837+0000","last_peered":"2024-09-05T13:16:53.125837+0000","last_clean":"2024-09-05T13:16:53.125837+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:53.125837+0000","last_undegraded":"2024-09-05T13:16:53.125837+0000","last_fullsized":"2024-09-05T13:16:53.125837+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7558,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11145","reported_seq":11283,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.080409+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:53.080409+0000","last_peered":"2024-09-05T13:16:53.080409+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:53.080409+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41000960,"num_objects":10010,"num_object_clones":0,"num_object_copies":20020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5559},{"shards":"0,2","objects":3886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10842","reported_seq":12487,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.142524+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:53.142524+0000","last_peered":"2024-09-05T13:16:53.142524+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:53.142524+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7161,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":753,"num_bytes_recovered":3084288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6103},{"shards":"0,2","objects":3682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11074","reported_seq":11206,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.038317+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:53.038317+0000","last_peered":"2024-09-05T13:16:53.038317+0000","last_clean":"2024-09-05T13:16:53.038317+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:53.038317+0000","last_undegraded":"2024-09-05T13:16:53.038317+0000","last_fullsized":"2024-09-05T13:16:53.038317+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41082880,"num_objects":10030,"num_object_clones":0,"num_object_copies":20060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11095","reported_seq":11230,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.031047+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:53.031047+0000","last_peered":"2024-09-05T13:16:53.031047+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:53.031047+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40951808,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8749,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3906},{"shards":"2","objects":5547}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10993","reported_seq":11131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.075114+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:53.075114+0000","last_peered":"2024-09-05T13:16:53.075114+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:53.075114+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620032,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7360,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3920}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11019","reported_seq":19920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:52.954371+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:52.954371+0000","last_peered":"2024-09-05T13:16:52.954371+0000","last_clean":"2024-09-05T13:16:52.954371+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:52.954371+0000","last_undegraded":"2024-09-05T13:16:52.954371+0000","last_fullsized":"2024-09-05T13:16:52.954371+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40517632,"num_objects":9892,"num_object_clones":0,"num_object_copies":19784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8614,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10806","reported_seq":10929,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.014337+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:53.014337+0000","last_peered":"2024-09-05T13:16:53.014337+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:53.014337+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40042496,"num_objects":9776,"num_object_clones":0,"num_object_copies":19552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8315,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3875},{"shards":"2","objects":5389}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10979","reported_seq":11119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.068788+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:53.068788+0000","last_peered":"2024-09-05T13:16:53.068788+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:53.068788+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546304,"num_objects":9899,"num_object_clones":0,"num_object_copies":19798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7130,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5480},{"shards":"0,2","objects":3880}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11111","reported_seq":11298,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.067090+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:53.067090+0000","last_peered":"2024-09-05T13:16:53.067090+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:53.067090+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40931328,"num_objects":9993,"num_object_clones":0,"num_object_copies":19986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7010,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5549},{"shards":"0,2","objects":3889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11098","reported_seq":11237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.113743+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:53.113743+0000","last_peered":"2024-09-05T13:16:53.113743+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:53.113743+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40886272,"num_objects":9982,"num_object_clones":0,"num_object_copies":19964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7228,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5416},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11120","reported_seq":11260,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.112776+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:53.112776+0000","last_peered":"2024-09-05T13:16:53.112776+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:53.112776+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41255395,"num_objects":10073,"num_object_clones":0,"num_object_copies":20146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7788,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5508},{"shards":"0,2","objects":4047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":651489787,"num_objects":159057,"num_object_clones":0,"num_object_copies":318114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":159057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132130,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5145,"num_bytes_recovered":21065728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1346494464,"data_stored":1346403221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754252,"kb_used_data":461564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93617588,"statfs":{"total":96636764160,"available":95864410112,"internally_reserved":0,"allocated":472641536,"data_stored":472381979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":891312,"kb_used_data":548696,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93480528,"statfs":{"total":96636764160,"available":95724060672,"internally_reserved":0,"allocated":561864704,"data_stored":561605142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,6,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581152,"kb_used_data":306512,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93790688,"statfs":{"total":96636764160,"available":96041664512,"internally_reserved":0,"allocated":313868288,"data_stored":313532394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561373184,"data_stored":561369571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313380864,"data_stored":313300919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471740416,"data_stored":471732731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:55.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:16:55.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10988", 'reported_seq': 11063, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:52.922851+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:52.922851+0000', 'last_peered': '2024-09-05T13:16:52.922851+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:52.922851+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40894464, 'num_objects': 9984, 'num_object_clones': 0, 'num_object_copies': 19968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10988, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5563}, {'shards': '0,2', 'objects': 3918}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:55.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:16:55.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11156", 'reported_seq': 11284, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:52.951954+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:52.951954+0000', 'last_peered': '2024-09-05T13:16:52.951954+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:52.951954+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41119768, 'num_objects': 10040, 'num_object_clones': 0, 'num_object_copies': 20080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10040, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8967, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3936}, {'shards': '2', 'objects': 5548}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:55.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:16:55.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11145", 'reported_seq': 11283, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.080409+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:53.080409+0000', 'last_peered': '2024-09-05T13:16:53.080409+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:53.080409+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41000960, 'num_objects': 10010, 'num_object_clones': 0, 'num_object_copies': 20020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7600, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5559}, {'shards': '0,2', 'objects': 3886}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:55.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:16:55.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10842", 'reported_seq': 12487, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.142524+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:53.142524+0000', 'last_peered': '2024-09-05T13:16:53.142524+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:53.142524+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40083456, 'num_objects': 9786, 'num_object_clones': 0, 'num_object_copies': 19572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 3682, 'num_objects_unfound': 0, 'num_objects_dirty': 9786, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7161, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 753, 'num_bytes_recovered': 3084288, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6103}, {'shards': '0,2', 'objects': 3682}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:55.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:16:55.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11095", 'reported_seq': 11230, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.031047+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:53.031047+0000', 'last_peered': '2024-09-05T13:16:53.031047+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:53.031047+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40951808, 'num_objects': 9998, 'num_object_clones': 0, 'num_object_copies': 19996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9998, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8749, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3906}, {'shards': '2', 'objects': 5547}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:55.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:16:55.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10993", 'reported_seq': 11131, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.075114+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:53.075114+0000', 'last_peered': '2024-09-05T13:16:53.075114+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:53.075114+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620032, 'num_objects': 9917, 'num_object_clones': 0, 'num_object_copies': 19834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9917, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7360, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5463}, {'shards': '0,2', 'objects': 3920}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:55.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:16:55.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10806", 'reported_seq': 10929, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.014337+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:53.014337+0000', 'last_peered': '2024-09-05T13:16:53.014337+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:53.014337+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40042496, 'num_objects': 9776, 'num_object_clones': 0, 'num_object_copies': 19552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8315, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3875}, {'shards': '2', 'objects': 5389}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:16:55.075 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:16:55.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10979", 'reported_seq': 11119, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.068788+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:53.068788+0000', 'last_peered': '2024-09-05T13:16:53.068788+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:53.068788+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40546304, 'num_objects': 9899, 'num_object_clones': 0, 'num_object_copies': 19798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9899, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7130, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5480}, {'shards': '0,2', 'objects': 3880}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:55.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:16:55.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11111", 'reported_seq': 11298, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.067090+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:53.067090+0000', 'last_peered': '2024-09-05T13:16:53.067090+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:53.067090+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40931328, 'num_objects': 9993, 'num_object_clones': 0, 'num_object_copies': 19986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9993, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7010, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5549}, {'shards': '0,2', 'objects': 3889}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:55.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:16:55.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11098", 'reported_seq': 11237, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.113743+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:53.113743+0000', 'last_peered': '2024-09-05T13:16:53.113743+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:53.113743+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40886272, 'num_objects': 9982, 'num_object_clones': 0, 'num_object_copies': 19964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7228, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5416}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:55.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:16:55.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11120", 'reported_seq': 11260, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:53.112776+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:53.112776+0000', 'last_peered': '2024-09-05T13:16:53.112776+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:53.112776+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41255395, 'num_objects': 10073, 'num_object_clones': 0, 'num_object_copies': 20146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10073, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7788, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5508}, {'shards': '0,2', 'objects': 4047}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:16:55.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:16:55.762 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:55.762 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/318118 objects degraded (14.042%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":342,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":159059,"data_bytes":651489806,"bytes_used":2280157184,"bytes_avail":287630135296,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":318118,"degraded_ratio":0.14041644924210514,"misplaced_objects":3682,"misplaced_total":318118,"misplaced_ratio":0.011574321478193626,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":57663,"recovering_keys_per_sec":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":364},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:16:55.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:16:55.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:56.360 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:56.380 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-05T13:16:55.609240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651489806,"num_objects":159059,"num_object_clones":0,"num_object_copies":318118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":159059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132132,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5146,"num_bytes_recovered":21065747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2226716,"kb_used_data":1316772,"kb_used_omap":12,"kb_used_meta":909875,"kb_avail":280888804,"statfs":{"total":289910292480,"available":287630135296,"internally_reserved":0,"allocated":1348374528,"data_stored":1347519515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":931712006},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,9,7,8,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17936384,"num_objects":-4379,"num_object_clones":0,"num_object_copies":-8758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-169,"num_objects_unfound":0,"num_objects_dirty":-4379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4379,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004513"},"pg_stats":[{"pgid":"2.f","version":"45'10988","reported_seq":11063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:52.922851+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:52.922851+0000","last_peered":"2024-09-05T13:16:52.922851+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:52.922851+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10988,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5563},{"shards":"0,2","objects":3918}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10922","reported_seq":11027,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:52.951595+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:52.951595+0000","last_peered":"2024-09-05T13:16:52.951595+0000","last_clean":"2024-09-05T13:16:52.951595+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:52.951595+0000","last_undegraded":"2024-09-05T13:16:52.951595+0000","last_fullsized":"2024-09-05T13:16:52.951595+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40271872,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8932,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11156","reported_seq":11284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:52.951954+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:52.951954+0000","last_peered":"2024-09-05T13:16:52.951954+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:52.951954+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41119768,"num_objects":10040,"num_object_clones":0,"num_object_copies":20080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8967,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3936},{"shards":"2","objects":5548}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11046","reported_seq":11116,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.023882+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:53.023882+0000","last_peered":"2024-09-05T13:16:53.023882+0000","last_clean":"2024-09-05T13:16:53.023882+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:53.023882+0000","last_undegraded":"2024-09-05T13:16:53.023882+0000","last_fullsized":"2024-09-05T13:16:53.023882+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878080,"num_objects":9980,"num_object_clones":0,"num_object_copies":19960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11046,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11018","reported_seq":11152,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.125837+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:53.125837+0000","last_peered":"2024-09-05T13:16:53.125837+0000","last_clean":"2024-09-05T13:16:53.125837+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:53.125837+0000","last_undegraded":"2024-09-05T13:16:53.125837+0000","last_fullsized":"2024-09-05T13:16:53.125837+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7558,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11145","reported_seq":11283,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.080409+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:53.080409+0000","last_peered":"2024-09-05T13:16:53.080409+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:53.080409+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41000960,"num_objects":10010,"num_object_clones":0,"num_object_copies":20020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5559},{"shards":"0,2","objects":3886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10842","reported_seq":12487,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.142524+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:53.142524+0000","last_peered":"2024-09-05T13:16:53.142524+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:53.142524+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7161,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":753,"num_bytes_recovered":3084288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6103},{"shards":"0,2","objects":3682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11074","reported_seq":11206,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:53.038317+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:53.038317+0000","last_peered":"2024-09-05T13:16:53.038317+0000","last_clean":"2024-09-05T13:16:53.038317+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:53.038317+0000","last_undegraded":"2024-09-05T13:16:53.038317+0000","last_fullsized":"2024-09-05T13:16:53.038317+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41082880,"num_objects":10030,"num_object_clones":0,"num_object_copies":20060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11095","reported_seq":11230,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.031047+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:53.031047+0000","last_peered":"2024-09-05T13:16:53.031047+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:53.031047+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40951808,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8749,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3906},{"shards":"2","objects":5547}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10993","reported_seq":11131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.075114+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:53.075114+0000","last_peered":"2024-09-05T13:16:53.075114+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:53.075114+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620032,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7360,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3920}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11019","reported_seq":19920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:52.954371+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:52.954371+0000","last_peered":"2024-09-05T13:16:52.954371+0000","last_clean":"2024-09-05T13:16:52.954371+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:52.954371+0000","last_undegraded":"2024-09-05T13:16:52.954371+0000","last_fullsized":"2024-09-05T13:16:52.954371+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40517632,"num_objects":9892,"num_object_clones":0,"num_object_copies":19784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8614,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10806","reported_seq":10929,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.014337+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:53.014337+0000","last_peered":"2024-09-05T13:16:53.014337+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:53.014337+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40042496,"num_objects":9776,"num_object_clones":0,"num_object_copies":19552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8315,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3875},{"shards":"2","objects":5389}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10979","reported_seq":11119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.068788+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:53.068788+0000","last_peered":"2024-09-05T13:16:53.068788+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:53.068788+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546304,"num_objects":9899,"num_object_clones":0,"num_object_copies":19798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7130,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5480},{"shards":"0,2","objects":3880}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11111","reported_seq":11298,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.067090+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:53.067090+0000","last_peered":"2024-09-05T13:16:53.067090+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:53.067090+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40931328,"num_objects":9993,"num_object_clones":0,"num_object_copies":19986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7010,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5549},{"shards":"0,2","objects":3889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11098","reported_seq":11237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.113743+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:53.113743+0000","last_peered":"2024-09-05T13:16:53.113743+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:53.113743+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40886272,"num_objects":9982,"num_object_clones":0,"num_object_copies":19964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7228,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5416},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11120","reported_seq":11260,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:53.112776+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:53.112776+0000","last_peered":"2024-09-05T13:16:53.112776+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:53.112776+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41255395,"num_objects":10073,"num_object_clones":0,"num_object_copies":20146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7788,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5508},{"shards":"0,2","objects":4047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":651489787,"num_objects":159057,"num_object_clones":0,"num_object_copies":318114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3682,"num_objects_unfound":0,"num_objects_dirty":159057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132130,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5145,"num_bytes_recovered":21065728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1346494464,"data_stored":1346403221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754252,"kb_used_data":461564,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93617588,"statfs":{"total":96636764160,"available":95864410112,"internally_reserved":0,"allocated":472641536,"data_stored":472381979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":891312,"kb_used_data":548696,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93480528,"statfs":{"total":96636764160,"available":95724060672,"internally_reserved":0,"allocated":561864704,"data_stored":561605142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,6,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581152,"kb_used_data":306512,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93790688,"statfs":{"total":96636764160,"available":96041664512,"internally_reserved":0,"allocated":313868288,"data_stored":313532394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561373184,"data_stored":561369571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313380864,"data_stored":313300919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471740416,"data_stored":471732731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:59.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:16:59.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:16:59.942 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:16:59.966 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-05T13:16:59.610634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642617870,"num_objects":156893,"num_object_clones":0,"num_object_copies":313786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":156893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134298,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5240,"num_bytes_recovered":21450771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194408,"kb_used_data":1305260,"kb_used_omap":12,"kb_used_meta":889075,"kb_avail":280921112,"statfs":{"total":289910292480,"available":287663218688,"internally_reserved":0,"allocated":1336586240,"data_stored":1335731227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":910412806},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,9,4,6,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26808320,"num_objects":-6545,"num_object_clones":0,"num_object_copies":-13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":-6545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6545,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004206"},"pg_stats":[{"pgid":"2.f","version":"45'11127","reported_seq":11202,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.220202+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:58.220202+0000","last_peered":"2024-09-05T13:16:58.220202+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:58.220202+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11127,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5424},{"shards":"0,2","objects":3779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11056","reported_seq":11161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.001197+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:58.001197+0000","last_peered":"2024-09-05T13:16:58.001197+0000","last_clean":"2024-09-05T13:16:58.001197+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:58.001197+0000","last_undegraded":"2024-09-05T13:16:58.001197+0000","last_fullsized":"2024-09-05T13:16:58.001197+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39723008,"num_objects":9698,"num_object_clones":0,"num_object_copies":19396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9066,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11315","reported_seq":11442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.005690+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:58.005690+0000","last_peered":"2024-09-05T13:16:58.005690+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:58.005690+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40468504,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9126,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3777},{"shards":"2","objects":5389}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11167","reported_seq":11237,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.031794+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:58.031794+0000","last_peered":"2024-09-05T13:16:58.031794+0000","last_clean":"2024-09-05T13:16:58.031794+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:58.031794+0000","last_undegraded":"2024-09-05T13:16:58.031794+0000","last_fullsized":"2024-09-05T13:16:58.031794+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40382464,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11167,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11144","reported_seq":11278,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.079960+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:58.079960+0000","last_peered":"2024-09-05T13:16:58.079960+0000","last_clean":"2024-09-05T13:16:58.079960+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:58.079960+0000","last_undegraded":"2024-09-05T13:16:58.079960+0000","last_fullsized":"2024-09-05T13:16:58.079960+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11285","reported_seq":11423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:57.972430+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:57.972430+0000","last_peered":"2024-09-05T13:16:57.972430+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:57.972430+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5419},{"shards":"0,2","objects":3746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10967","reported_seq":12798,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.197831+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:58.197831+0000","last_peered":"2024-09-05T13:16:58.197831+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:58.197831+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39571456,"num_objects":9661,"num_object_clones":0,"num_object_copies":19322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":9661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":3469312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6072},{"shards":"0,2","objects":3588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11193","reported_seq":11325,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.180074+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:58.180074+0000","last_peered":"2024-09-05T13:16:58.180074+0000","last_clean":"2024-09-05T13:16:58.180074+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:58.180074+0000","last_undegraded":"2024-09-05T13:16:58.180074+0000","last_fullsized":"2024-09-05T13:16:58.180074+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595456,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7803,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11223","reported_seq":11358,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.029281+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:58.029281+0000","last_peered":"2024-09-05T13:16:58.029281+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:58.029281+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8877,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3778},{"shards":"2","objects":5419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11128","reported_seq":11264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.229990+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:58.229990+0000","last_peered":"2024-09-05T13:16:58.229990+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:58.229990+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40067072,"num_objects":9782,"num_object_clones":0,"num_object_copies":19564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7495,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5328},{"shards":"0,2","objects":3785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11167","reported_seq":20068,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:57.974320+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:57.974320+0000","last_peered":"2024-09-05T13:16:57.974320+0000","last_clean":"2024-09-05T13:16:57.974320+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:57.974320+0000","last_undegraded":"2024-09-05T13:16:57.974320+0000","last_fullsized":"2024-09-05T13:16:57.974320+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39911424,"num_objects":9744,"num_object_clones":0,"num_object_copies":19488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10937","reported_seq":11060,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.030003+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:58.030003+0000","last_peered":"2024-09-05T13:16:58.030003+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:58.030003+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39505920,"num_objects":9645,"num_object_clones":0,"num_object_copies":19290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8446,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3744},{"shards":"2","objects":5258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11117","reported_seq":11257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.121092+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:58.121092+0000","last_peered":"2024-09-05T13:16:58.121092+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:58.121092+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":3742}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11240","reported_seq":11424,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.144837+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:58.144837+0000","last_peered":"2024-09-05T13:16:58.144837+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:58.144837+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7139,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5420},{"shards":"0,2","objects":3760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11231","reported_seq":11370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.219593+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:58.219593+0000","last_peered":"2024-09-05T13:16:58.219593+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:58.219593+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40341504,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7361,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5283},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11281","reported_seq":11421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.103073+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:58.103073+0000","last_peered":"2024-09-05T13:16:58.103073+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:58.103073+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595939,"num_objects":9912,"num_object_clones":0,"num_object_copies":19824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":3886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642617851,"num_objects":156891,"num_object_clones":0,"num_object_copies":313782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":156891,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134296,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5239,"num_bytes_recovered":21450752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1334734848,"data_stored":1334643605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737456,"kb_used_data":457236,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93634384,"statfs":{"total":96636764160,"available":95881609216,"internally_reserved":0,"allocated":468209664,"data_stored":467950107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":889080,"kb_used_data":542312,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93482760,"statfs":{"total":96636764160,"available":95726346240,"internally_reserved":0,"allocated":555327488,"data_stored":555067926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567872,"kb_used_data":305712,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93803968,"statfs":{"total":96636764160,"available":96055263232,"internally_reserved":0,"allocated":313049088,"data_stored":312713194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268431023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554778624,"data_stored":554775011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312573952,"data_stored":312494007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467382272,"data_stored":467374587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:16:59.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:00.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:00.592 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-05T13:16:59.610634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642617870,"num_objects":156893,"num_object_clones":0,"num_object_copies":313786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":156893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134298,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5240,"num_bytes_recovered":21450771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194408,"kb_used_data":1305260,"kb_used_omap":12,"kb_used_meta":889075,"kb_avail":280921112,"statfs":{"total":289910292480,"available":287663218688,"internally_reserved":0,"allocated":1336586240,"data_stored":1335731227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":910412806},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,9,4,6,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26808320,"num_objects":-6545,"num_object_clones":0,"num_object_copies":-13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":-6545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6545,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004206"},"pg_stats":[{"pgid":"2.f","version":"45'11127","reported_seq":11202,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.220202+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:58.220202+0000","last_peered":"2024-09-05T13:16:58.220202+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:58.220202+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11127,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5424},{"shards":"0,2","objects":3779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11056","reported_seq":11161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.001197+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:58.001197+0000","last_peered":"2024-09-05T13:16:58.001197+0000","last_clean":"2024-09-05T13:16:58.001197+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:58.001197+0000","last_undegraded":"2024-09-05T13:16:58.001197+0000","last_fullsized":"2024-09-05T13:16:58.001197+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39723008,"num_objects":9698,"num_object_clones":0,"num_object_copies":19396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9066,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11315","reported_seq":11442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.005690+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:58.005690+0000","last_peered":"2024-09-05T13:16:58.005690+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:58.005690+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40468504,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9126,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3777},{"shards":"2","objects":5389}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11167","reported_seq":11237,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.031794+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:58.031794+0000","last_peered":"2024-09-05T13:16:58.031794+0000","last_clean":"2024-09-05T13:16:58.031794+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:58.031794+0000","last_undegraded":"2024-09-05T13:16:58.031794+0000","last_fullsized":"2024-09-05T13:16:58.031794+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40382464,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11167,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11144","reported_seq":11278,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.079960+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:58.079960+0000","last_peered":"2024-09-05T13:16:58.079960+0000","last_clean":"2024-09-05T13:16:58.079960+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:58.079960+0000","last_undegraded":"2024-09-05T13:16:58.079960+0000","last_fullsized":"2024-09-05T13:16:58.079960+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11285","reported_seq":11423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:57.972430+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:57.972430+0000","last_peered":"2024-09-05T13:16:57.972430+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:57.972430+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5419},{"shards":"0,2","objects":3746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10967","reported_seq":12798,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.197831+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:58.197831+0000","last_peered":"2024-09-05T13:16:58.197831+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:58.197831+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39571456,"num_objects":9661,"num_object_clones":0,"num_object_copies":19322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":9661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":3469312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6072},{"shards":"0,2","objects":3588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11193","reported_seq":11325,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.180074+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:58.180074+0000","last_peered":"2024-09-05T13:16:58.180074+0000","last_clean":"2024-09-05T13:16:58.180074+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:58.180074+0000","last_undegraded":"2024-09-05T13:16:58.180074+0000","last_fullsized":"2024-09-05T13:16:58.180074+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595456,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7803,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11223","reported_seq":11358,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.029281+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:58.029281+0000","last_peered":"2024-09-05T13:16:58.029281+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:58.029281+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8877,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3778},{"shards":"2","objects":5419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11128","reported_seq":11264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.229990+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:58.229990+0000","last_peered":"2024-09-05T13:16:58.229990+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:58.229990+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40067072,"num_objects":9782,"num_object_clones":0,"num_object_copies":19564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7495,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5328},{"shards":"0,2","objects":3785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11167","reported_seq":20068,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:57.974320+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:57.974320+0000","last_peered":"2024-09-05T13:16:57.974320+0000","last_clean":"2024-09-05T13:16:57.974320+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:57.974320+0000","last_undegraded":"2024-09-05T13:16:57.974320+0000","last_fullsized":"2024-09-05T13:16:57.974320+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39911424,"num_objects":9744,"num_object_clones":0,"num_object_copies":19488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10937","reported_seq":11060,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.030003+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:58.030003+0000","last_peered":"2024-09-05T13:16:58.030003+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:58.030003+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39505920,"num_objects":9645,"num_object_clones":0,"num_object_copies":19290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8446,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3744},{"shards":"2","objects":5258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11117","reported_seq":11257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.121092+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:58.121092+0000","last_peered":"2024-09-05T13:16:58.121092+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:58.121092+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":3742}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11240","reported_seq":11424,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.144837+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:58.144837+0000","last_peered":"2024-09-05T13:16:58.144837+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:58.144837+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7139,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5420},{"shards":"0,2","objects":3760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11231","reported_seq":11370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.219593+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:58.219593+0000","last_peered":"2024-09-05T13:16:58.219593+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:58.219593+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40341504,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7361,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5283},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11281","reported_seq":11421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.103073+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:58.103073+0000","last_peered":"2024-09-05T13:16:58.103073+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:58.103073+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595939,"num_objects":9912,"num_object_clones":0,"num_object_copies":19824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":3886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642617851,"num_objects":156891,"num_object_clones":0,"num_object_copies":313782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":156891,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134296,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5239,"num_bytes_recovered":21450752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1334734848,"data_stored":1334643605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737456,"kb_used_data":457236,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93634384,"statfs":{"total":96636764160,"available":95881609216,"internally_reserved":0,"allocated":468209664,"data_stored":467950107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":889080,"kb_used_data":542312,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93482760,"statfs":{"total":96636764160,"available":95726346240,"internally_reserved":0,"allocated":555327488,"data_stored":555067926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567872,"kb_used_data":305712,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93803968,"statfs":{"total":96636764160,"available":96055263232,"internally_reserved":0,"allocated":313049088,"data_stored":312713194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268431023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554778624,"data_stored":554775011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312573952,"data_stored":312494007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467382272,"data_stored":467374587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:00.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:00.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11127", 'reported_seq': 11202, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.220202+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:16:58.220202+0000', 'last_peered': '2024-09-05T13:16:58.220202+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:16:58.220202+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40325120, 'num_objects': 9845, 'num_object_clones': 0, 'num_object_copies': 19690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11127, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5424}, {'shards': '0,2', 'objects': 3779}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11315", 'reported_seq': 11442, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.005690+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:16:58.005690+0000', 'last_peered': '2024-09-05T13:16:58.005690+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:16:58.005690+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40468504, 'num_objects': 9881, 'num_object_clones': 0, 'num_object_copies': 19762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9881, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9126, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3777}, {'shards': '2', 'objects': 5389}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11285", 'reported_seq': 11423, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:57.972430+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:16:57.972430+0000', 'last_peered': '2024-09-05T13:16:57.972430+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:16:57.972430+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40427520, 'num_objects': 9870, 'num_object_clones': 0, 'num_object_copies': 19740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7740, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5419}, {'shards': '0,2', 'objects': 3746}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10967", 'reported_seq': 12798, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.197831+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:16:58.197831+0000', 'last_peered': '2024-09-05T13:16:58.197831+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:16:58.197831+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39571456, 'num_objects': 9661, 'num_object_clones': 0, 'num_object_copies': 19322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 3588, 'num_objects_unfound': 0, 'num_objects_dirty': 9661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7286, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 847, 'num_bytes_recovered': 3469312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6072}, {'shards': '0,2', 'objects': 3588}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11223", 'reported_seq': 11358, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.029281+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:16:58.029281+0000', 'last_peered': '2024-09-05T13:16:58.029281+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:16:58.029281+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40427520, 'num_objects': 9870, 'num_object_clones': 0, 'num_object_copies': 19740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8877, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3778}, {'shards': '2', 'objects': 5419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:00.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11128", 'reported_seq': 11264, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.229990+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:16:58.229990+0000', 'last_peered': '2024-09-05T13:16:58.229990+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:16:58.229990+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40067072, 'num_objects': 9782, 'num_object_clones': 0, 'num_object_copies': 19564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7495, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5328}, {'shards': '0,2', 'objects': 3785}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'10937", 'reported_seq': 11060, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.030003+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:16:58.030003+0000', 'last_peered': '2024-09-05T13:16:58.030003+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:16:58.030003+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39505920, 'num_objects': 9645, 'num_object_clones': 0, 'num_object_copies': 19290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9645, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8446, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3744}, {'shards': '2', 'objects': 5258}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11117", 'reported_seq': 11257, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.121092+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:16:58.121092+0000', 'last_peered': '2024-09-05T13:16:58.121092+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:16:58.121092+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39981056, 'num_objects': 9761, 'num_object_clones': 0, 'num_object_copies': 19522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9761, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7268, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5342}, {'shards': '0,2', 'objects': 3742}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11240", 'reported_seq': 11424, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.144837+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:16:58.144837+0000', 'last_peered': '2024-09-05T13:16:58.144837+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:16:58.144837+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40402944, 'num_objects': 9864, 'num_object_clones': 0, 'num_object_copies': 19728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7139, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5420}, {'shards': '0,2', 'objects': 3760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11231", 'reported_seq': 11370, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.219593+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:16:58.219593+0000', 'last_peered': '2024-09-05T13:16:58.219593+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:16:58.219593+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40341504, 'num_objects': 9849, 'num_object_clones': 0, 'num_object_copies': 19698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9849, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7361, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5283}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:00.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11281", 'reported_seq': 11421, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:16:58.103073+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:16:58.103073+0000', 'last_peered': '2024-09-05T13:16:58.103073+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:16:58.103073+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40595939, 'num_objects': 9912, 'num_object_clones': 0, 'num_object_copies': 19824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7949, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5347}, {'shards': '0,2', 'objects': 3886}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:00.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:01.330 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:01.330 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/313786 objects degraded (14.235%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":348,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156893,"data_bytes":642617870,"bytes_used":2247073792,"bytes_avail":287663218688,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":313786,"degraded_ratio":0.14235498078308145,"misplaced_objects":3588,"misplaced_total":313786,"misplaced_ratio":0.011434544562217562,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":89739,"recovering_keys_per_sec":0,"num_objects_recovered":263,"num_bytes_recovered":1077248,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":545},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:01.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:01.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:01.939 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:01.941 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:01.961 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-05T13:17:01.611512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642617870,"num_objects":156893,"num_object_clones":0,"num_object_copies":313786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":156893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134298,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5240,"num_bytes_recovered":21450771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2201972,"kb_used_data":1304504,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280913548,"statfs":{"total":289910292480,"available":287655473152,"internally_reserved":0,"allocated":1335812096,"data_stored":1334957083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":918932486},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,12,4,7,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17903616,"num_objects":-4371,"num_object_clones":0,"num_object_copies":-8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-174,"num_objects_unfound":0,"num_objects_dirty":-4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":712704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004271"},"pg_stats":[{"pgid":"2.f","version":"45'11127","reported_seq":11202,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.220202+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:16:58.220202+0000","last_peered":"2024-09-05T13:16:58.220202+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:16:58.220202+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11127,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5424},{"shards":"0,2","objects":3779}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11056","reported_seq":11161,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.001197+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:16:58.001197+0000","last_peered":"2024-09-05T13:16:58.001197+0000","last_clean":"2024-09-05T13:16:58.001197+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:16:58.001197+0000","last_undegraded":"2024-09-05T13:16:58.001197+0000","last_fullsized":"2024-09-05T13:16:58.001197+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39723008,"num_objects":9698,"num_object_clones":0,"num_object_copies":19396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9066,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11315","reported_seq":11442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.005690+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:16:58.005690+0000","last_peered":"2024-09-05T13:16:58.005690+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:16:58.005690+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40468504,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9126,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3777},{"shards":"2","objects":5389}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11167","reported_seq":11237,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.031794+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:16:58.031794+0000","last_peered":"2024-09-05T13:16:58.031794+0000","last_clean":"2024-09-05T13:16:58.031794+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:16:58.031794+0000","last_undegraded":"2024-09-05T13:16:58.031794+0000","last_fullsized":"2024-09-05T13:16:58.031794+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40382464,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11167,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11144","reported_seq":11278,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.079960+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:16:58.079960+0000","last_peered":"2024-09-05T13:16:58.079960+0000","last_clean":"2024-09-05T13:16:58.079960+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:16:58.079960+0000","last_undegraded":"2024-09-05T13:16:58.079960+0000","last_fullsized":"2024-09-05T13:16:58.079960+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7684,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11285","reported_seq":11423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:57.972430+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:16:57.972430+0000","last_peered":"2024-09-05T13:16:57.972430+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:16:57.972430+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5419},{"shards":"0,2","objects":3746}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10967","reported_seq":12798,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.197831+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:16:58.197831+0000","last_peered":"2024-09-05T13:16:58.197831+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:16:58.197831+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39571456,"num_objects":9661,"num_object_clones":0,"num_object_copies":19322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":9661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":3469312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6072},{"shards":"0,2","objects":3588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11193","reported_seq":11325,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:58.180074+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:16:58.180074+0000","last_peered":"2024-09-05T13:16:58.180074+0000","last_clean":"2024-09-05T13:16:58.180074+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:16:58.180074+0000","last_undegraded":"2024-09-05T13:16:58.180074+0000","last_fullsized":"2024-09-05T13:16:58.180074+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595456,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7803,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11223","reported_seq":11358,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.029281+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:16:58.029281+0000","last_peered":"2024-09-05T13:16:58.029281+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:16:58.029281+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8877,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3778},{"shards":"2","objects":5419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11128","reported_seq":11264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.229990+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:16:58.229990+0000","last_peered":"2024-09-05T13:16:58.229990+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:16:58.229990+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40067072,"num_objects":9782,"num_object_clones":0,"num_object_copies":19564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7495,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5328},{"shards":"0,2","objects":3785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11167","reported_seq":20068,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:57.974320+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:16:57.974320+0000","last_peered":"2024-09-05T13:16:57.974320+0000","last_clean":"2024-09-05T13:16:57.974320+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:16:57.974320+0000","last_undegraded":"2024-09-05T13:16:57.974320+0000","last_fullsized":"2024-09-05T13:16:57.974320+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39911424,"num_objects":9744,"num_object_clones":0,"num_object_copies":19488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10937","reported_seq":11060,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.030003+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:16:58.030003+0000","last_peered":"2024-09-05T13:16:58.030003+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:16:58.030003+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39505920,"num_objects":9645,"num_object_clones":0,"num_object_copies":19290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8446,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3744},{"shards":"2","objects":5258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11117","reported_seq":11257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.121092+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:16:58.121092+0000","last_peered":"2024-09-05T13:16:58.121092+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:16:58.121092+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7268,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5342},{"shards":"0,2","objects":3742}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11240","reported_seq":11424,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.144837+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:16:58.144837+0000","last_peered":"2024-09-05T13:16:58.144837+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:16:58.144837+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7139,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5420},{"shards":"0,2","objects":3760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11231","reported_seq":11370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.219593+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:16:58.219593+0000","last_peered":"2024-09-05T13:16:58.219593+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:16:58.219593+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40341504,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7361,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5283},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11281","reported_seq":11421,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:16:58.103073+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:16:58.103073+0000","last_peered":"2024-09-05T13:16:58.103073+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:16:58.103073+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595939,"num_objects":9912,"num_object_clones":0,"num_object_copies":19824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":3886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642617851,"num_objects":156891,"num_object_clones":0,"num_object_copies":313782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3588,"num_objects_unfound":0,"num_objects_dirty":156891,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":134296,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5239,"num_bytes_recovered":21450752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1333903360,"data_stored":1333812117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737456,"kb_used_data":457236,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93634384,"statfs":{"total":96636764160,"available":95881609216,"internally_reserved":0,"allocated":468209664,"data_stored":467950107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":889080,"kb_used_data":542312,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93482760,"statfs":{"total":96636764160,"available":95726346240,"internally_reserved":0,"allocated":555327488,"data_stored":555067926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575436,"kb_used_data":304956,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93796404,"statfs":{"total":96636764160,"available":96047517696,"internally_reserved":0,"allocated":312274944,"data_stored":311939050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276950703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554778624,"data_stored":554775011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311742464,"data_stored":311662519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":467382272,"data_stored":467374587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:04.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:05.508 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:05.510 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:05.527 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-05T13:17:03.614137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633795086,"num_objects":154739,"num_object_clones":0,"num_object_copies":309478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":154739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136452,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5324,"num_bytes_recovered":21794835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160835,"ondisk_log_size":160835,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2201048,"kb_used_data":1293860,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":280914472,"statfs":{"total":289910292480,"available":287656419328,"internally_reserved":0,"allocated":1324912640,"data_stored":1324057627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":928893958},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,8,10,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26726400,"num_objects":-6525,"num_object_clones":0,"num_object_copies":-13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-258,"num_objects_unfound":0,"num_objects_dirty":-6525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6525,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006202"},"pg_stats":[{"pgid":"2.f","version":"45'11251","reported_seq":11326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.233331+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:03.233331+0000","last_peered":"2024-09-05T13:17:03.233331+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:03.233331+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39817216,"num_objects":9721,"num_object_clones":0,"num_object_copies":19442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11251,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5300},{"shards":"0,2","objects":3655}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11159","reported_seq":11264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.902350+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:02.902350+0000","last_peered":"2024-09-05T13:17:02.902350+0000","last_clean":"2024-09-05T13:17:02.902350+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:02.902350+0000","last_undegraded":"2024-09-05T13:17:02.902350+0000","last_fullsized":"2024-09-05T13:17:02.902350+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301120,"num_objects":9595,"num_object_clones":0,"num_object_copies":19190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11456","reported_seq":11581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.059686+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:03.059686+0000","last_peered":"2024-09-05T13:17:03.059686+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:03.059686+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890968,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9267,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3636},{"shards":"2","objects":5248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11285","reported_seq":11355,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.983484+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:02.983484+0000","last_peered":"2024-09-05T13:17:02.983484+0000","last_clean":"2024-09-05T13:17:02.983484+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:02.983484+0000","last_undegraded":"2024-09-05T13:17:02.983484+0000","last_fullsized":"2024-09-05T13:17:02.983484+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11285,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11294","reported_seq":11428,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:03.220569+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:03.220569+0000","last_peered":"2024-09-05T13:17:03.220569+0000","last_clean":"2024-09-05T13:17:03.220569+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:03.220569+0000","last_undegraded":"2024-09-05T13:17:03.220569+0000","last_fullsized":"2024-09-05T13:17:03.220569+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39276544,"num_objects":9589,"num_object_clones":0,"num_object_copies":19178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7834,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11413","reported_seq":11551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.216672+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:03.216672+0000","last_peered":"2024-09-05T13:17:03.216672+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:03.216672+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7868,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5291},{"shards":"0,2","objects":3618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11105","reported_seq":13108,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.225922+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:03.225922+0000","last_peered":"2024-09-05T13:17:03.225922+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:03.225922+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39006208,"num_objects":9523,"num_object_clones":0,"num_object_copies":19046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":9523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7424,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":931,"num_bytes_recovered":3813376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6018},{"shards":"0,2","objects":3504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11331","reported_seq":11463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:03.118965+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:03.118965+0000","last_peered":"2024-09-05T13:17:03.118965+0000","last_clean":"2024-09-05T13:17:03.118965+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:03.118965+0000","last_undegraded":"2024-09-05T13:17:03.118965+0000","last_fullsized":"2024-09-05T13:17:03.118965+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40030208,"num_objects":9773,"num_object_clones":0,"num_object_copies":19546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11371","reported_seq":11505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.040930+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:03.040930+0000","last_peered":"2024-09-05T13:17:03.040930+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:03.040930+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39821312,"num_objects":9722,"num_object_clones":0,"num_object_copies":19444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9025,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3630},{"shards":"2","objects":5271}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11268","reported_seq":11406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.216930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:03.216930+0000","last_peered":"2024-09-05T13:17:03.216930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:03.216930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39493632,"num_objects":9642,"num_object_clones":0,"num_object_copies":19284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7635,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5188},{"shards":"0,2","objects":3645}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11307","reported_seq":20208,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.996109+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:02.996109+0000","last_peered":"2024-09-05T13:17:02.996109+0000","last_clean":"2024-09-05T13:17:02.996109+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:02.996109+0000","last_undegraded":"2024-09-05T13:17:02.996109+0000","last_fullsized":"2024-09-05T13:17:02.996109+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8902,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11063","reported_seq":11186,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.031170+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:03.031170+0000","last_peered":"2024-09-05T13:17:03.031170+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:03.031170+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38989824,"num_objects":9519,"num_object_clones":0,"num_object_copies":19038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8572,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3618},{"shards":"2","objects":5132}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11282","reported_seq":11422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.157844+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:03.157844+0000","last_peered":"2024-09-05T13:17:03.157844+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:03.157844+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39305216,"num_objects":9596,"num_object_clones":0,"num_object_copies":19192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7433,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5177},{"shards":"0,2","objects":3577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11365","reported_seq":11552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.190581+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:03.190581+0000","last_peered":"2024-09-05T13:17:03.190581+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:03.190581+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7264,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":3635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11368","reported_seq":11507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.235721+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:03.235721+0000","last_peered":"2024-09-05T13:17:03.235721+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:03.235721+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39780352,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7498,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5146},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11414","reported_seq":11554,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.047116+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:03.047116+0000","last_peered":"2024-09-05T13:17:03.047116+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:03.047116+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40051171,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8082,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5214},{"shards":"0,2","objects":3753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633795067,"num_objects":154737,"num_object_clones":0,"num_object_copies":309474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":154737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136450,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5323,"num_bytes_recovered":21794816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1323008000,"data_stored":1322916757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737460,"kb_used_data":453092,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93634380,"statfs":{"total":96636764160,"available":95881605120,"internally_reserved":0,"allocated":463966208,"data_stored":463706651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888152,"kb_used_data":535812,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93483688,"statfs":{"total":96636764160,"available":95727296512,"internally_reserved":0,"allocated":548671488,"data_stored":548411926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,7,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575436,"kb_used_data":304956,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93796404,"statfs":{"total":96636764160,"available":96047517696,"internally_reserved":0,"allocated":312274944,"data_stored":311939050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276950703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548241408,"data_stored":548237795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311742464,"data_stored":311662519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463024128,"data_stored":463016443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:05.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:06.078 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:06.104 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-05T13:17:05.614811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633795086,"num_objects":154739,"num_object_clones":0,"num_object_copies":309478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":154739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136452,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5324,"num_bytes_recovered":21794835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160835,"ondisk_log_size":160835,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2201048,"kb_used_data":1293860,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":280914472,"statfs":{"total":289910292480,"available":287656419328,"internally_reserved":0,"allocated":1324912640,"data_stored":1324057627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":928893958},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,8,10,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17694720,"num_objects":-4320,"num_object_clones":0,"num_object_copies":-8640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-178,"num_objects_unfound":0,"num_objects_dirty":-4320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":729088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006203"},"pg_stats":[{"pgid":"2.f","version":"45'11251","reported_seq":11326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.233331+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:03.233331+0000","last_peered":"2024-09-05T13:17:03.233331+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:03.233331+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39817216,"num_objects":9721,"num_object_clones":0,"num_object_copies":19442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11251,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5300},{"shards":"0,2","objects":3655}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11159","reported_seq":11264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.902350+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:02.902350+0000","last_peered":"2024-09-05T13:17:02.902350+0000","last_clean":"2024-09-05T13:17:02.902350+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:02.902350+0000","last_undegraded":"2024-09-05T13:17:02.902350+0000","last_fullsized":"2024-09-05T13:17:02.902350+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301120,"num_objects":9595,"num_object_clones":0,"num_object_copies":19190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11456","reported_seq":11581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.059686+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:03.059686+0000","last_peered":"2024-09-05T13:17:03.059686+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:03.059686+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890968,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9267,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3636},{"shards":"2","objects":5248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11285","reported_seq":11355,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.983484+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:02.983484+0000","last_peered":"2024-09-05T13:17:02.983484+0000","last_clean":"2024-09-05T13:17:02.983484+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:02.983484+0000","last_undegraded":"2024-09-05T13:17:02.983484+0000","last_fullsized":"2024-09-05T13:17:02.983484+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11285,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11294","reported_seq":11428,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:03.220569+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:03.220569+0000","last_peered":"2024-09-05T13:17:03.220569+0000","last_clean":"2024-09-05T13:17:03.220569+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:03.220569+0000","last_undegraded":"2024-09-05T13:17:03.220569+0000","last_fullsized":"2024-09-05T13:17:03.220569+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39276544,"num_objects":9589,"num_object_clones":0,"num_object_copies":19178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7834,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11413","reported_seq":11551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.216672+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:03.216672+0000","last_peered":"2024-09-05T13:17:03.216672+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:03.216672+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7868,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5291},{"shards":"0,2","objects":3618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11105","reported_seq":13108,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.225922+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:03.225922+0000","last_peered":"2024-09-05T13:17:03.225922+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:03.225922+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39006208,"num_objects":9523,"num_object_clones":0,"num_object_copies":19046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":9523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7424,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":931,"num_bytes_recovered":3813376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6018},{"shards":"0,2","objects":3504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11331","reported_seq":11463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:03.118965+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:03.118965+0000","last_peered":"2024-09-05T13:17:03.118965+0000","last_clean":"2024-09-05T13:17:03.118965+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:03.118965+0000","last_undegraded":"2024-09-05T13:17:03.118965+0000","last_fullsized":"2024-09-05T13:17:03.118965+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40030208,"num_objects":9773,"num_object_clones":0,"num_object_copies":19546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11371","reported_seq":11505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.040930+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:03.040930+0000","last_peered":"2024-09-05T13:17:03.040930+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:03.040930+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39821312,"num_objects":9722,"num_object_clones":0,"num_object_copies":19444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9025,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3630},{"shards":"2","objects":5271}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11268","reported_seq":11406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.216930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:03.216930+0000","last_peered":"2024-09-05T13:17:03.216930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:03.216930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39493632,"num_objects":9642,"num_object_clones":0,"num_object_copies":19284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7635,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5188},{"shards":"0,2","objects":3645}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11307","reported_seq":20208,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.996109+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:02.996109+0000","last_peered":"2024-09-05T13:17:02.996109+0000","last_clean":"2024-09-05T13:17:02.996109+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:02.996109+0000","last_undegraded":"2024-09-05T13:17:02.996109+0000","last_fullsized":"2024-09-05T13:17:02.996109+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8902,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11063","reported_seq":11186,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.031170+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:03.031170+0000","last_peered":"2024-09-05T13:17:03.031170+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:03.031170+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38989824,"num_objects":9519,"num_object_clones":0,"num_object_copies":19038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8572,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3618},{"shards":"2","objects":5132}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11282","reported_seq":11422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.157844+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:03.157844+0000","last_peered":"2024-09-05T13:17:03.157844+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:03.157844+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39305216,"num_objects":9596,"num_object_clones":0,"num_object_copies":19192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7433,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5177},{"shards":"0,2","objects":3577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11365","reported_seq":11552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.190581+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:03.190581+0000","last_peered":"2024-09-05T13:17:03.190581+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:03.190581+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7264,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":3635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11368","reported_seq":11507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.235721+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:03.235721+0000","last_peered":"2024-09-05T13:17:03.235721+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:03.235721+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39780352,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7498,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5146},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11414","reported_seq":11554,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.047116+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:03.047116+0000","last_peered":"2024-09-05T13:17:03.047116+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:03.047116+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40051171,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8082,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5214},{"shards":"0,2","objects":3753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633795067,"num_objects":154737,"num_object_clones":0,"num_object_copies":309474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":154737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136450,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5323,"num_bytes_recovered":21794816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1323008000,"data_stored":1322916757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737460,"kb_used_data":453092,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93634380,"statfs":{"total":96636764160,"available":95881605120,"internally_reserved":0,"allocated":463966208,"data_stored":463706651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888152,"kb_used_data":535812,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93483688,"statfs":{"total":96636764160,"available":95727296512,"internally_reserved":0,"allocated":548671488,"data_stored":548411926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,7,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575436,"kb_used_data":304956,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93796404,"statfs":{"total":96636764160,"available":96047517696,"internally_reserved":0,"allocated":312274944,"data_stored":311939050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276950703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548241408,"data_stored":548237795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311742464,"data_stored":311662519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463024128,"data_stored":463016443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:06.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:06.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11251", 'reported_seq': 11326, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.233331+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:03.233331+0000', 'last_peered': '2024-09-05T13:17:03.233331+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:03.233331+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39817216, 'num_objects': 9721, 'num_object_clones': 0, 'num_object_copies': 19442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11251, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5300}, {'shards': '0,2', 'objects': 3655}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11456", 'reported_seq': 11581, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.059686+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:03.059686+0000', 'last_peered': '2024-09-05T13:17:03.059686+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:03.059686+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39890968, 'num_objects': 9740, 'num_object_clones': 0, 'num_object_copies': 19480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9740, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9267, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3636}, {'shards': '2', 'objects': 5248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11413", 'reported_seq': 11551, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.216672+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:03.216672+0000', 'last_peered': '2024-09-05T13:17:03.216672+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:03.216672+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39903232, 'num_objects': 9742, 'num_object_clones': 0, 'num_object_copies': 19484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7868, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5291}, {'shards': '0,2', 'objects': 3618}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11105", 'reported_seq': 13108, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.225922+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:03.225922+0000', 'last_peered': '2024-09-05T13:17:03.225922+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:03.225922+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39006208, 'num_objects': 9523, 'num_object_clones': 0, 'num_object_copies': 19046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 3504, 'num_objects_unfound': 0, 'num_objects_dirty': 9523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7424, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 931, 'num_bytes_recovered': 3813376, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6018}, {'shards': '0,2', 'objects': 3504}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11371", 'reported_seq': 11505, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.040930+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:03.040930+0000', 'last_peered': '2024-09-05T13:17:03.040930+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:03.040930+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39821312, 'num_objects': 9722, 'num_object_clones': 0, 'num_object_copies': 19444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9025, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3630}, {'shards': '2', 'objects': 5271}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:06.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11268", 'reported_seq': 11406, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.216930+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:03.216930+0000', 'last_peered': '2024-09-05T13:17:03.216930+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:03.216930+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39493632, 'num_objects': 9642, 'num_object_clones': 0, 'num_object_copies': 19284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9642, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7635, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5188}, {'shards': '0,2', 'objects': 3645}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11063", 'reported_seq': 11186, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.031170+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:03.031170+0000', 'last_peered': '2024-09-05T13:17:03.031170+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:03.031170+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38989824, 'num_objects': 9519, 'num_object_clones': 0, 'num_object_copies': 19038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9519, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8572, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3618}, {'shards': '2', 'objects': 5132}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11282", 'reported_seq': 11422, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.157844+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:03.157844+0000', 'last_peered': '2024-09-05T13:17:03.157844+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:03.157844+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39305216, 'num_objects': 9596, 'num_object_clones': 0, 'num_object_copies': 19192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7433, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5177}, {'shards': '0,2', 'objects': 3577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11365", 'reported_seq': 11552, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.190581+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:03.190581+0000', 'last_peered': '2024-09-05T13:17:03.190581+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:03.190581+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39890944, 'num_objects': 9739, 'num_object_clones': 0, 'num_object_copies': 19478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7264, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5295}, {'shards': '0,2', 'objects': 3635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11368", 'reported_seq': 11507, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.235721+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:03.235721+0000', 'last_peered': '2024-09-05T13:17:03.235721+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:03.235721+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39780352, 'num_objects': 9712, 'num_object_clones': 0, 'num_object_copies': 19424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9712, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7498, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5146}, {'shards': '0,2', 'objects': 3740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:06.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11414", 'reported_seq': 11554, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:03.047116+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:03.047116+0000', 'last_peered': '2024-09-05T13:17:03.047116+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:03.047116+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40051171, 'num_objects': 9779, 'num_object_clones': 0, 'num_object_copies': 19558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8082, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5214}, {'shards': '0,2', 'objects': 3753}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:06.108 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:06.808 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:06.808 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/309478 objects degraded (14.434%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":353,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":154739,"data_bytes":633795086,"bytes_used":2253873152,"bytes_avail":287656419328,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":309478,"degraded_ratio":0.14433659258493334,"misplaced_objects":3504,"misplaced_total":309478,"misplaced_ratio":0.011322291083695771,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":60725,"recovering_keys_per_sec":0,"num_objects_recovered":178,"num_bytes_recovered":729088,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":359},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:06.834 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:06.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:07.367 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:07.369 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:07.391 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-05T13:17:05.614811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633795086,"num_objects":154739,"num_object_clones":0,"num_object_copies":309478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":154739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136452,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5324,"num_bytes_recovered":21794835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160835,"ondisk_log_size":160835,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2201048,"kb_used_data":1293860,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":280914472,"statfs":{"total":289910292480,"available":287656419328,"internally_reserved":0,"allocated":1324912640,"data_stored":1324057627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":928893958},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,8,10,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17694720,"num_objects":-4320,"num_object_clones":0,"num_object_copies":-8640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-178,"num_objects_unfound":0,"num_objects_dirty":-4320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":729088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006203"},"pg_stats":[{"pgid":"2.f","version":"45'11251","reported_seq":11326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.233331+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:03.233331+0000","last_peered":"2024-09-05T13:17:03.233331+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:03.233331+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39817216,"num_objects":9721,"num_object_clones":0,"num_object_copies":19442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11251,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5300},{"shards":"0,2","objects":3655}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11159","reported_seq":11264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.902350+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:02.902350+0000","last_peered":"2024-09-05T13:17:02.902350+0000","last_clean":"2024-09-05T13:17:02.902350+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:02.902350+0000","last_undegraded":"2024-09-05T13:17:02.902350+0000","last_fullsized":"2024-09-05T13:17:02.902350+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39301120,"num_objects":9595,"num_object_clones":0,"num_object_copies":19190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9169,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11456","reported_seq":11581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.059686+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:03.059686+0000","last_peered":"2024-09-05T13:17:03.059686+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:03.059686+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890968,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9267,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3636},{"shards":"2","objects":5248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11285","reported_seq":11355,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.983484+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:02.983484+0000","last_peered":"2024-09-05T13:17:02.983484+0000","last_clean":"2024-09-05T13:17:02.983484+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:02.983484+0000","last_undegraded":"2024-09-05T13:17:02.983484+0000","last_fullsized":"2024-09-05T13:17:02.983484+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11285,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11294","reported_seq":11428,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:03.220569+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:03.220569+0000","last_peered":"2024-09-05T13:17:03.220569+0000","last_clean":"2024-09-05T13:17:03.220569+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:03.220569+0000","last_undegraded":"2024-09-05T13:17:03.220569+0000","last_fullsized":"2024-09-05T13:17:03.220569+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39276544,"num_objects":9589,"num_object_clones":0,"num_object_copies":19178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7834,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11413","reported_seq":11551,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.216672+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:03.216672+0000","last_peered":"2024-09-05T13:17:03.216672+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:03.216672+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903232,"num_objects":9742,"num_object_clones":0,"num_object_copies":19484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7868,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5291},{"shards":"0,2","objects":3618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11105","reported_seq":13108,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.225922+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:03.225922+0000","last_peered":"2024-09-05T13:17:03.225922+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:03.225922+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39006208,"num_objects":9523,"num_object_clones":0,"num_object_copies":19046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":9523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7424,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":931,"num_bytes_recovered":3813376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6018},{"shards":"0,2","objects":3504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11331","reported_seq":11463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:03.118965+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:03.118965+0000","last_peered":"2024-09-05T13:17:03.118965+0000","last_clean":"2024-09-05T13:17:03.118965+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:03.118965+0000","last_undegraded":"2024-09-05T13:17:03.118965+0000","last_fullsized":"2024-09-05T13:17:03.118965+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40030208,"num_objects":9773,"num_object_clones":0,"num_object_copies":19546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11371","reported_seq":11505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.040930+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:03.040930+0000","last_peered":"2024-09-05T13:17:03.040930+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:03.040930+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39821312,"num_objects":9722,"num_object_clones":0,"num_object_copies":19444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9025,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3630},{"shards":"2","objects":5271}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11268","reported_seq":11406,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.216930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:03.216930+0000","last_peered":"2024-09-05T13:17:03.216930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:03.216930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39493632,"num_objects":9642,"num_object_clones":0,"num_object_copies":19284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7635,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5188},{"shards":"0,2","objects":3645}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11307","reported_seq":20208,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:02.996109+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:02.996109+0000","last_peered":"2024-09-05T13:17:02.996109+0000","last_clean":"2024-09-05T13:17:02.996109+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:02.996109+0000","last_undegraded":"2024-09-05T13:17:02.996109+0000","last_fullsized":"2024-09-05T13:17:02.996109+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8902,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11063","reported_seq":11186,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.031170+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:03.031170+0000","last_peered":"2024-09-05T13:17:03.031170+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:03.031170+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38989824,"num_objects":9519,"num_object_clones":0,"num_object_copies":19038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8572,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3618},{"shards":"2","objects":5132}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11282","reported_seq":11422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.157844+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:03.157844+0000","last_peered":"2024-09-05T13:17:03.157844+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:03.157844+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39305216,"num_objects":9596,"num_object_clones":0,"num_object_copies":19192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7433,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5177},{"shards":"0,2","objects":3577}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11365","reported_seq":11552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.190581+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:03.190581+0000","last_peered":"2024-09-05T13:17:03.190581+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:03.190581+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7264,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5295},{"shards":"0,2","objects":3635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11368","reported_seq":11507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.235721+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:03.235721+0000","last_peered":"2024-09-05T13:17:03.235721+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:03.235721+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39780352,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7498,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5146},{"shards":"0,2","objects":3740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11414","reported_seq":11554,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:03.047116+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:03.047116+0000","last_peered":"2024-09-05T13:17:03.047116+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:03.047116+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40051171,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8082,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5214},{"shards":"0,2","objects":3753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633795067,"num_objects":154737,"num_object_clones":0,"num_object_copies":309474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":154737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":136450,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5323,"num_bytes_recovered":21794816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1323008000,"data_stored":1322916757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737460,"kb_used_data":453092,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93634380,"statfs":{"total":96636764160,"available":95881605120,"internally_reserved":0,"allocated":463966208,"data_stored":463706651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":888152,"kb_used_data":535812,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93483688,"statfs":{"total":96636764160,"available":95727296512,"internally_reserved":0,"allocated":548671488,"data_stored":548411926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,7,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575436,"kb_used_data":304956,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93796404,"statfs":{"total":96636764160,"available":96047517696,"internally_reserved":0,"allocated":312274944,"data_stored":311939050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276950703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548241408,"data_stored":548237795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311742464,"data_stored":311662519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463024128,"data_stored":463016443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:10.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:10.958 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:10.958 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:10.986 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-05T13:17:09.616046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":624853518,"num_objects":152556,"num_object_clones":0,"num_object_copies":305112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":152556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138635,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5410,"num_bytes_recovered":22147091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2224088,"kb_used_data":1282208,"kb_used_omap":12,"kb_used_meta":941811,"kb_avail":280891432,"statfs":{"total":289910292480,"available":287632826368,"internally_reserved":0,"allocated":1312980992,"data_stored":1312125979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":964414470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,5,12,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26636288,"num_objects":-6503,"num_object_clones":0,"num_object_copies":-13006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":-6503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6503,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006192"},"pg_stats":[{"pgid":"2.f","version":"45'11381","reported_seq":11456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.144465+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:08.144465+0000","last_peered":"2024-09-05T13:17:08.144465+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:08.144465+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11381,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5170},{"shards":"0,2","objects":3525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11278","reported_seq":11383,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.001130+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:08.001130+0000","last_peered":"2024-09-05T13:17:08.001130+0000","last_clean":"2024-09-05T13:17:08.001130+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:08.001130+0000","last_undegraded":"2024-09-05T13:17:08.001130+0000","last_fullsized":"2024-09-05T13:17:08.001130+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38813696,"num_objects":9476,"num_object_clones":0,"num_object_copies":18952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9288,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11605","reported_seq":11733,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.077789+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:08.077789+0000","last_peered":"2024-09-05T13:17:08.077789+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:08.077789+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39280664,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9416,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3487},{"shards":"2","objects":5099}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11435","reported_seq":11505,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:07.916914+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:07.916914+0000","last_peered":"2024-09-05T13:17:07.916914+0000","last_clean":"2024-09-05T13:17:07.916914+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:07.916914+0000","last_undegraded":"2024-09-05T13:17:07.916914+0000","last_fullsized":"2024-09-05T13:17:07.916914+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11435,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11409","reported_seq":11543,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.245391+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:08.245391+0000","last_peered":"2024-09-05T13:17:08.245391+0000","last_clean":"2024-09-05T13:17:08.245391+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:08.245391+0000","last_undegraded":"2024-09-05T13:17:08.245391+0000","last_fullsized":"2024-09-05T13:17:08.245391+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38805504,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11545","reported_seq":11682,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.242263+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:08.242263+0000","last_peered":"2024-09-05T13:17:08.242263+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:08.242263+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39362560,"num_objects":9610,"num_object_clones":0,"num_object_copies":19220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8000,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11236","reported_seq":13411,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.267428+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:08.267428+0000","last_peered":"2024-09-05T13:17:08.267428+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:08.267428+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469632,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7555,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1017,"num_bytes_recovered":4165632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5973},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11483","reported_seq":11615,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.242044+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:08.242044+0000","last_peered":"2024-09-05T13:17:08.242044+0000","last_clean":"2024-09-05T13:17:08.242044+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:08.242044+0000","last_undegraded":"2024-09-05T13:17:08.242044+0000","last_fullsized":"2024-09-05T13:17:08.242044+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8093,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11519","reported_seq":11654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:07.958937+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:07.958937+0000","last_peered":"2024-09-05T13:17:07.958937+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:07.958937+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9173,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3482},{"shards":"2","objects":5123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11398","reported_seq":11536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.217679+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:08.217679+0000","last_peered":"2024-09-05T13:17:08.217679+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:08.217679+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961152,"num_objects":9512,"num_object_clones":0,"num_object_copies":19024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7765,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11461","reported_seq":20360,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.041932+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:08.041932+0000","last_peered":"2024-09-05T13:17:08.041932+0000","last_clean":"2024-09-05T13:17:08.041932+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:08.041932+0000","last_undegraded":"2024-09-05T13:17:08.041932+0000","last_fullsized":"2024-09-05T13:17:08.041932+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9056,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11201","reported_seq":11324,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.073720+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:08.073720+0000","last_peered":"2024-09-05T13:17:08.073720+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:08.073720+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38424576,"num_objects":9381,"num_object_clones":0,"num_object_copies":18762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8710,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3480},{"shards":"2","objects":4994}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11418","reported_seq":11558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.204334+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:08.204334+0000","last_peered":"2024-09-05T13:17:08.204334+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:08.204334+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38748160,"num_objects":9460,"num_object_clones":0,"num_object_copies":18920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7569,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5041},{"shards":"0,2","objects":3441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11494","reported_seq":11679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.232416+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:08.232416+0000","last_peered":"2024-09-05T13:17:08.232416+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:08.232416+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39362560,"num_objects":9610,"num_object_clones":0,"num_object_copies":19220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7393,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5166},{"shards":"0,2","objects":3506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11491","reported_seq":11630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.272157+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:08.272157+0000","last_peered":"2024-09-05T13:17:08.272157+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:08.272157+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39276544,"num_objects":9589,"num_object_clones":0,"num_object_copies":19178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3617}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11561","reported_seq":11701,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.196095+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:08.196095+0000","last_peered":"2024-09-05T13:17:08.196095+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:08.196095+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39449059,"num_objects":9632,"num_object_clones":0,"num_object_copies":19264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5067},{"shards":"0,2","objects":3606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":624853499,"num_objects":152554,"num_object_clones":0,"num_object_copies":305108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":152554,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138633,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5409,"num_bytes_recovered":22147072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1311096832,"data_stored":1311005589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737164,"kb_used_data":448636,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93634676,"statfs":{"total":96636764160,"available":95881908224,"internally_reserved":0,"allocated":459403264,"data_stored":459143707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908128,"kb_used_data":529420,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93463712,"statfs":{"total":96636764160,"available":95706841088,"internally_reserved":0,"allocated":542126080,"data_stored":541866518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578796,"kb_used_data":304152,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93793044,"statfs":{"total":96636764160,"available":96044077056,"internally_reserved":0,"allocated":311451648,"data_stored":311115754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541696000,"data_stored":541692387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310972416,"data_stored":310892471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":458428416,"data_stored":458420731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:10.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:11.554 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:11.556 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:11.580 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-05T13:17:09.616046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":624853518,"num_objects":152556,"num_object_clones":0,"num_object_copies":305112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":152556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138635,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5410,"num_bytes_recovered":22147091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2224088,"kb_used_data":1282208,"kb_used_omap":12,"kb_used_meta":941811,"kb_avail":280891432,"statfs":{"total":289910292480,"available":287632826368,"internally_reserved":0,"allocated":1312980992,"data_stored":1312125979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":964414470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,5,12,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26636288,"num_objects":-6503,"num_object_clones":0,"num_object_copies":-13006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":-6503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6503,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006192"},"pg_stats":[{"pgid":"2.f","version":"45'11381","reported_seq":11456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.144465+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:08.144465+0000","last_peered":"2024-09-05T13:17:08.144465+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:08.144465+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11381,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5170},{"shards":"0,2","objects":3525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11278","reported_seq":11383,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.001130+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:08.001130+0000","last_peered":"2024-09-05T13:17:08.001130+0000","last_clean":"2024-09-05T13:17:08.001130+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:08.001130+0000","last_undegraded":"2024-09-05T13:17:08.001130+0000","last_fullsized":"2024-09-05T13:17:08.001130+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38813696,"num_objects":9476,"num_object_clones":0,"num_object_copies":18952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9288,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11605","reported_seq":11733,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.077789+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:08.077789+0000","last_peered":"2024-09-05T13:17:08.077789+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:08.077789+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39280664,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9416,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3487},{"shards":"2","objects":5099}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11435","reported_seq":11505,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:07.916914+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:07.916914+0000","last_peered":"2024-09-05T13:17:07.916914+0000","last_clean":"2024-09-05T13:17:07.916914+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:07.916914+0000","last_undegraded":"2024-09-05T13:17:07.916914+0000","last_fullsized":"2024-09-05T13:17:07.916914+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11435,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11409","reported_seq":11543,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.245391+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:08.245391+0000","last_peered":"2024-09-05T13:17:08.245391+0000","last_clean":"2024-09-05T13:17:08.245391+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:08.245391+0000","last_undegraded":"2024-09-05T13:17:08.245391+0000","last_fullsized":"2024-09-05T13:17:08.245391+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38805504,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11545","reported_seq":11682,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.242263+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:08.242263+0000","last_peered":"2024-09-05T13:17:08.242263+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:08.242263+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39362560,"num_objects":9610,"num_object_clones":0,"num_object_copies":19220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8000,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11236","reported_seq":13411,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.267428+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:08.267428+0000","last_peered":"2024-09-05T13:17:08.267428+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:08.267428+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469632,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7555,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1017,"num_bytes_recovered":4165632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5973},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11483","reported_seq":11615,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.242044+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:08.242044+0000","last_peered":"2024-09-05T13:17:08.242044+0000","last_clean":"2024-09-05T13:17:08.242044+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:08.242044+0000","last_undegraded":"2024-09-05T13:17:08.242044+0000","last_fullsized":"2024-09-05T13:17:08.242044+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8093,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11519","reported_seq":11654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:07.958937+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:07.958937+0000","last_peered":"2024-09-05T13:17:07.958937+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:07.958937+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9173,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3482},{"shards":"2","objects":5123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11398","reported_seq":11536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.217679+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:08.217679+0000","last_peered":"2024-09-05T13:17:08.217679+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:08.217679+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961152,"num_objects":9512,"num_object_clones":0,"num_object_copies":19024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7765,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11461","reported_seq":20360,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.041932+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:08.041932+0000","last_peered":"2024-09-05T13:17:08.041932+0000","last_clean":"2024-09-05T13:17:08.041932+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:08.041932+0000","last_undegraded":"2024-09-05T13:17:08.041932+0000","last_fullsized":"2024-09-05T13:17:08.041932+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9056,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11201","reported_seq":11324,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.073720+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:08.073720+0000","last_peered":"2024-09-05T13:17:08.073720+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:08.073720+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38424576,"num_objects":9381,"num_object_clones":0,"num_object_copies":18762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8710,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3480},{"shards":"2","objects":4994}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11418","reported_seq":11558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.204334+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:08.204334+0000","last_peered":"2024-09-05T13:17:08.204334+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:08.204334+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38748160,"num_objects":9460,"num_object_clones":0,"num_object_copies":18920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7569,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5041},{"shards":"0,2","objects":3441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11494","reported_seq":11679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.232416+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:08.232416+0000","last_peered":"2024-09-05T13:17:08.232416+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:08.232416+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39362560,"num_objects":9610,"num_object_clones":0,"num_object_copies":19220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7393,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5166},{"shards":"0,2","objects":3506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11491","reported_seq":11630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.272157+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:08.272157+0000","last_peered":"2024-09-05T13:17:08.272157+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:08.272157+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39276544,"num_objects":9589,"num_object_clones":0,"num_object_copies":19178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3617}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11561","reported_seq":11701,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.196095+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:08.196095+0000","last_peered":"2024-09-05T13:17:08.196095+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:08.196095+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39449059,"num_objects":9632,"num_object_clones":0,"num_object_copies":19264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5067},{"shards":"0,2","objects":3606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":624853499,"num_objects":152554,"num_object_clones":0,"num_object_copies":305108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":152554,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138633,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5409,"num_bytes_recovered":22147072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1311096832,"data_stored":1311005589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737164,"kb_used_data":448636,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93634676,"statfs":{"total":96636764160,"available":95881908224,"internally_reserved":0,"allocated":459403264,"data_stored":459143707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908128,"kb_used_data":529420,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93463712,"statfs":{"total":96636764160,"available":95706841088,"internally_reserved":0,"allocated":542126080,"data_stored":541866518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578796,"kb_used_data":304152,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93793044,"statfs":{"total":96636764160,"available":96044077056,"internally_reserved":0,"allocated":311451648,"data_stored":311115754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541696000,"data_stored":541692387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310972416,"data_stored":310892471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":458428416,"data_stored":458420731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:11.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:11.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11381", 'reported_seq': 11456, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.144465+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:08.144465+0000', 'last_peered': '2024-09-05T13:17:08.144465+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:08.144465+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39284736, 'num_objects': 9591, 'num_object_clones': 0, 'num_object_copies': 19182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11381, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5170}, {'shards': '0,2', 'objects': 3525}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:11.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:11.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11605", 'reported_seq': 11733, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.077789+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:08.077789+0000', 'last_peered': '2024-09-05T13:17:08.077789+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:08.077789+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39280664, 'num_objects': 9591, 'num_object_clones': 0, 'num_object_copies': 19182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9591, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9416, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3487}, {'shards': '2', 'objects': 5099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:11.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11545", 'reported_seq': 11682, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.242263+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:08.242263+0000', 'last_peered': '2024-09-05T13:17:08.242263+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:08.242263+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39362560, 'num_objects': 9610, 'num_object_clones': 0, 'num_object_copies': 19220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9610, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8000, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5159}, {'shards': '0,2', 'objects': 3486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11236", 'reported_seq': 13411, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.267428+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:08.267428+0000', 'last_peered': '2024-09-05T13:17:08.267428+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:08.267428+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38469632, 'num_objects': 9392, 'num_object_clones': 0, 'num_object_copies': 18784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 3418, 'num_objects_unfound': 0, 'num_objects_dirty': 9392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7555, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1017, 'num_bytes_recovered': 4165632, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5973}, {'shards': '0,2', 'objects': 3418}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11519", 'reported_seq': 11654, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:07.958937+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:07.958937+0000', 'last_peered': '2024-09-05T13:17:07.958937+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:07.958937+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39215104, 'num_objects': 9574, 'num_object_clones': 0, 'num_object_copies': 19148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9173, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3482}, {'shards': '2', 'objects': 5123}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11398", 'reported_seq': 11536, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.217679+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:08.217679+0000', 'last_peered': '2024-09-05T13:17:08.217679+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:08.217679+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38961152, 'num_objects': 9512, 'num_object_clones': 0, 'num_object_copies': 19024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7765, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5058}, {'shards': '0,2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11201", 'reported_seq': 11324, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.073720+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:08.073720+0000', 'last_peered': '2024-09-05T13:17:08.073720+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:08.073720+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38424576, 'num_objects': 9381, 'num_object_clones': 0, 'num_object_copies': 18762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9381, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8710, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3480}, {'shards': '2', 'objects': 4994}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:11.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11418", 'reported_seq': 11558, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.204334+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:08.204334+0000', 'last_peered': '2024-09-05T13:17:08.204334+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:08.204334+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38748160, 'num_objects': 9460, 'num_object_clones': 0, 'num_object_copies': 18920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7569, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5041}, {'shards': '0,2', 'objects': 3441}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11494", 'reported_seq': 11679, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.232416+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:08.232416+0000', 'last_peered': '2024-09-05T13:17:08.232416+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:08.232416+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39362560, 'num_objects': 9610, 'num_object_clones': 0, 'num_object_copies': 19220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9610, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7393, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5166}, {'shards': '0,2', 'objects': 3506}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11491", 'reported_seq': 11630, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.272157+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:08.272157+0000', 'last_peered': '2024-09-05T13:17:08.272157+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:08.272157+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39276544, 'num_objects': 9589, 'num_object_clones': 0, 'num_object_copies': 19178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9589, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7621, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5023}, {'shards': '0,2', 'objects': 3617}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:11.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11561", 'reported_seq': 11701, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:08.196095+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:08.196095+0000', 'last_peered': '2024-09-05T13:17:08.196095+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:08.196095+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39449059, 'num_objects': 9632, 'num_object_clones': 0, 'num_object_copies': 19264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8229, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5067}, {'shards': '0,2', 'objects': 3606}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:11.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:12.273 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:12.273 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/305112 objects degraded (14.640%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":359,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152556,"data_bytes":624853518,"bytes_used":2263982080,"bytes_avail":287646310400,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":305112,"degraded_ratio":0.14640197697894544,"misplaced_objects":3418,"misplaced_total":305112,"misplaced_ratio":0.011202443692807887,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":57992,"recovering_keys_per_sec":0,"num_objects_recovered":170,"num_bytes_recovered":696320,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":361},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:12.300 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:12.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:12.876 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:12.904 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-05T13:17:11.617799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":624853518,"num_objects":152556,"num_object_clones":0,"num_object_copies":305112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":152556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138635,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5410,"num_bytes_recovered":22147091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2210920,"kb_used_data":1281524,"kb_used_omap":12,"kb_used_meta":929331,"kb_avail":280904600,"statfs":{"total":289910292480,"available":287646310400,"internally_reserved":0,"allocated":1312280576,"data_stored":1311425563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":951634950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,3,5,13,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17764352,"num_objects":-4337,"num_object_clones":0,"num_object_copies":-8674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-170,"num_objects_unfound":0,"num_objects_dirty":-4337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4337,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":696320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007165"},"pg_stats":[{"pgid":"2.f","version":"45'11381","reported_seq":11456,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.144465+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:08.144465+0000","last_peered":"2024-09-05T13:17:08.144465+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:08.144465+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11381,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5170},{"shards":"0,2","objects":3525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11278","reported_seq":11383,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.001130+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:08.001130+0000","last_peered":"2024-09-05T13:17:08.001130+0000","last_clean":"2024-09-05T13:17:08.001130+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:08.001130+0000","last_undegraded":"2024-09-05T13:17:08.001130+0000","last_fullsized":"2024-09-05T13:17:08.001130+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38813696,"num_objects":9476,"num_object_clones":0,"num_object_copies":18952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9288,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11605","reported_seq":11733,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.077789+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:08.077789+0000","last_peered":"2024-09-05T13:17:08.077789+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:08.077789+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39280664,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9416,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3487},{"shards":"2","objects":5099}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11435","reported_seq":11505,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:07.916914+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:07.916914+0000","last_peered":"2024-09-05T13:17:07.916914+0000","last_clean":"2024-09-05T13:17:07.916914+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:07.916914+0000","last_undegraded":"2024-09-05T13:17:07.916914+0000","last_fullsized":"2024-09-05T13:17:07.916914+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11435,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11409","reported_seq":11543,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.245391+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:08.245391+0000","last_peered":"2024-09-05T13:17:08.245391+0000","last_clean":"2024-09-05T13:17:08.245391+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:08.245391+0000","last_undegraded":"2024-09-05T13:17:08.245391+0000","last_fullsized":"2024-09-05T13:17:08.245391+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38805504,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7949,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11545","reported_seq":11682,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.242263+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:08.242263+0000","last_peered":"2024-09-05T13:17:08.242263+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:08.242263+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39362560,"num_objects":9610,"num_object_clones":0,"num_object_copies":19220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8000,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":3486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11236","reported_seq":13411,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.267428+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:08.267428+0000","last_peered":"2024-09-05T13:17:08.267428+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:08.267428+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469632,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7555,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1017,"num_bytes_recovered":4165632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5973},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11483","reported_seq":11615,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.242044+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:08.242044+0000","last_peered":"2024-09-05T13:17:08.242044+0000","last_clean":"2024-09-05T13:17:08.242044+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:08.242044+0000","last_undegraded":"2024-09-05T13:17:08.242044+0000","last_fullsized":"2024-09-05T13:17:08.242044+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8093,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11519","reported_seq":11654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:07.958937+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:07.958937+0000","last_peered":"2024-09-05T13:17:07.958937+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:07.958937+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9173,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3482},{"shards":"2","objects":5123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11398","reported_seq":11536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.217679+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:08.217679+0000","last_peered":"2024-09-05T13:17:08.217679+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:08.217679+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38961152,"num_objects":9512,"num_object_clones":0,"num_object_copies":19024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7765,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11461","reported_seq":20360,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:08.041932+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:08.041932+0000","last_peered":"2024-09-05T13:17:08.041932+0000","last_clean":"2024-09-05T13:17:08.041932+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:08.041932+0000","last_undegraded":"2024-09-05T13:17:08.041932+0000","last_fullsized":"2024-09-05T13:17:08.041932+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9056,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11201","reported_seq":11324,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.073720+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:08.073720+0000","last_peered":"2024-09-05T13:17:08.073720+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:08.073720+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38424576,"num_objects":9381,"num_object_clones":0,"num_object_copies":18762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8710,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3480},{"shards":"2","objects":4994}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11418","reported_seq":11558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.204334+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:08.204334+0000","last_peered":"2024-09-05T13:17:08.204334+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:08.204334+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38748160,"num_objects":9460,"num_object_clones":0,"num_object_copies":18920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7569,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5041},{"shards":"0,2","objects":3441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11494","reported_seq":11679,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.232416+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:08.232416+0000","last_peered":"2024-09-05T13:17:08.232416+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:08.232416+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39362560,"num_objects":9610,"num_object_clones":0,"num_object_copies":19220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7393,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5166},{"shards":"0,2","objects":3506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11491","reported_seq":11630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.272157+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:08.272157+0000","last_peered":"2024-09-05T13:17:08.272157+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:08.272157+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39276544,"num_objects":9589,"num_object_clones":0,"num_object_copies":19178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7621,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3617}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11561","reported_seq":11701,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:08.196095+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:08.196095+0000","last_peered":"2024-09-05T13:17:08.196095+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:08.196095+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39449059,"num_objects":9632,"num_object_clones":0,"num_object_copies":19264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8229,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5067},{"shards":"0,2","objects":3606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":624853499,"num_objects":152554,"num_object_clones":0,"num_object_copies":305108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3418,"num_objects_unfound":0,"num_objects_dirty":152554,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138633,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5409,"num_bytes_recovered":22147072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1310375936,"data_stored":1310284693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":737164,"kb_used_data":448636,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93634676,"statfs":{"total":96636764160,"available":95881908224,"internally_reserved":0,"allocated":459403264,"data_stored":459143707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908128,"kb_used_data":529420,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93463712,"statfs":{"total":96636764160,"available":95706841088,"internally_reserved":0,"allocated":542126080,"data_stored":541866518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565628,"kb_used_data":303468,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93806212,"statfs":{"total":96636764160,"available":96057561088,"internally_reserved":0,"allocated":310751232,"data_stored":310415338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268431023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541696000,"data_stored":541692387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310251520,"data_stored":310171575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":458428416,"data_stored":458420731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:15.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:16.468 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:16.498 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-05T13:17:15.618887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616006158,"num_objects":150396,"num_object_clones":0,"num_object_copies":300792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":150396,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140795,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5497,"num_bytes_recovered":22503443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2209620,"kb_used_data":1271908,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":280905900,"statfs":{"total":289910292480,"available":287647641600,"internally_reserved":0,"allocated":1302433792,"data_stored":1301578779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":960154630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,1,2,6,13,12,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17788928,"num_objects":-4343,"num_object_clones":0,"num_object_copies":-8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":-4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4343,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004750"},"pg_stats":[{"pgid":"2.f","version":"45'11517","reported_seq":11592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.239814+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:13.239814+0000","last_peered":"2024-09-05T13:17:13.239814+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:13.239814+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38727680,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11517,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5034},{"shards":"0,2","objects":3389}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11397","reported_seq":11502,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.086931+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:13.086931+0000","last_peered":"2024-09-05T13:17:13.086931+0000","last_clean":"2024-09-05T13:17:13.086931+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:13.086931+0000","last_undegraded":"2024-09-05T13:17:13.086931+0000","last_fullsized":"2024-09-05T13:17:13.086931+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9407,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11739","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.083094+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:13.083094+0000","last_peered":"2024-09-05T13:17:13.083094+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:13.083094+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731800,"num_objects":9457,"num_object_clones":0,"num_object_copies":18914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9550,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3353},{"shards":"2","objects":4965}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11560","reported_seq":11630,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.069084+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:13.069084+0000","last_peered":"2024-09-05T13:17:13.069084+0000","last_clean":"2024-09-05T13:17:13.069084+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:13.069084+0000","last_undegraded":"2024-09-05T13:17:13.069084+0000","last_fullsized":"2024-09-05T13:17:13.069084+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38772736,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11560,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11557","reported_seq":11691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.107385+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:13.107385+0000","last_peered":"2024-09-05T13:17:13.107385+0000","last_clean":"2024-09-05T13:17:13.107385+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:13.107385+0000","last_undegraded":"2024-09-05T13:17:13.107385+0000","last_fullsized":"2024-09-05T13:17:13.107385+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8097,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11672","reported_seq":11810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.198812+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:13.198812+0000","last_peered":"2024-09-05T13:17:13.198812+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:13.198812+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8127,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11363","reported_seq":13712,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.290352+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:13.290352+0000","last_peered":"2024-09-05T13:17:13.290352+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:13.290352+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37949440,"num_objects":9265,"num_object_clones":0,"num_object_copies":18530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":9265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7682,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":4521984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5933},{"shards":"0,2","objects":3331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11621","reported_seq":11753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.265694+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:13.265694+0000","last_peered":"2024-09-05T13:17:13.265694+0000","last_clean":"2024-09-05T13:17:13.265694+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:13.265694+0000","last_undegraded":"2024-09-05T13:17:13.265694+0000","last_fullsized":"2024-09-05T13:17:13.265694+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8231,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11651","reported_seq":11786,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:12.935240+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:12.935240+0000","last_peered":"2024-09-05T13:17:12.935240+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:12.935240+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9305,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4991}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11532","reported_seq":11670,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.241767+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:13.241767+0000","last_peered":"2024-09-05T13:17:13.241767+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:13.241767+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7899,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4924},{"shards":"0,2","objects":3381}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11588","reported_seq":20489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.092195+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:13.092195+0000","last_peered":"2024-09-05T13:17:13.092195+0000","last_clean":"2024-09-05T13:17:13.092195+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:13.092195+0000","last_undegraded":"2024-09-05T13:17:13.092195+0000","last_fullsized":"2024-09-05T13:17:13.092195+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9183,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11347","reported_seq":11470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.040020+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:13.040020+0000","last_peered":"2024-09-05T13:17:13.040020+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:13.040020+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37826560,"num_objects":9235,"num_object_clones":0,"num_object_copies":18470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8856,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3334},{"shards":"2","objects":4848}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11562","reported_seq":11702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.174231+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:13.174231+0000","last_peered":"2024-09-05T13:17:13.174231+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:13.174231+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38158336,"num_objects":9316,"num_object_clones":0,"num_object_copies":18632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4897},{"shards":"0,2","objects":3297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11650","reported_seq":11837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.258888+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:13.258888+0000","last_peered":"2024-09-05T13:17:13.258888+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:13.258888+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38723584,"num_objects":9454,"num_object_clones":0,"num_object_copies":18908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7549,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5010},{"shards":"0,2","objects":3350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11613","reported_seq":11752,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.301069+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:13.301069+0000","last_peered":"2024-09-05T13:17:13.301069+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:13.301069+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7743,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4901},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11706","reported_seq":11844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.220182+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:13.220182+0000","last_peered":"2024-09-05T13:17:13.220182+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:13.220182+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38855139,"num_objects":9487,"num_object_clones":0,"num_object_copies":18974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8374,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4922},{"shards":"0,2","objects":3461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":616006139,"num_objects":150394,"num_object_clones":0,"num_object_copies":300788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":150394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140793,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5496,"num_bytes_recovered":22503424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1299337216,"data_stored":1299245973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736948,"kb_used_data":444260,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93634892,"statfs":{"total":96636764160,"available":95882129408,"internally_reserved":0,"allocated":454922240,"data_stored":454662683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907044,"kb_used_data":524180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93464796,"statfs":{"total":96636764160,"available":95707951104,"internally_reserved":0,"allocated":536760320,"data_stored":536500758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,0,3,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565628,"kb_used_data":303468,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93806212,"statfs":{"total":96636764160,"available":96057561088,"internally_reserved":0,"allocated":310751232,"data_stored":310415338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268431023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":535060480,"data_stored":535056867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310251520,"data_stored":310171575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454025216,"data_stored":454017531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:16.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:17.049 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:17.051 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:17.075 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-05T13:17:15.618887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616006158,"num_objects":150396,"num_object_clones":0,"num_object_copies":300792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":150396,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140795,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5497,"num_bytes_recovered":22503443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2209620,"kb_used_data":1271908,"kb_used_omap":12,"kb_used_meta":937651,"kb_avail":280905900,"statfs":{"total":289910292480,"available":287647641600,"internally_reserved":0,"allocated":1302433792,"data_stored":1301578779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":960154630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,1,2,6,13,12,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17788928,"num_objects":-4343,"num_object_clones":0,"num_object_copies":-8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":-4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4343,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004750"},"pg_stats":[{"pgid":"2.f","version":"45'11517","reported_seq":11592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.239814+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:13.239814+0000","last_peered":"2024-09-05T13:17:13.239814+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:13.239814+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38727680,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11517,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5034},{"shards":"0,2","objects":3389}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11397","reported_seq":11502,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.086931+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:13.086931+0000","last_peered":"2024-09-05T13:17:13.086931+0000","last_clean":"2024-09-05T13:17:13.086931+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:13.086931+0000","last_undegraded":"2024-09-05T13:17:13.086931+0000","last_fullsized":"2024-09-05T13:17:13.086931+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9407,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11739","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.083094+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:13.083094+0000","last_peered":"2024-09-05T13:17:13.083094+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:13.083094+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731800,"num_objects":9457,"num_object_clones":0,"num_object_copies":18914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9550,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3353},{"shards":"2","objects":4965}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11560","reported_seq":11630,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.069084+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:13.069084+0000","last_peered":"2024-09-05T13:17:13.069084+0000","last_clean":"2024-09-05T13:17:13.069084+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:13.069084+0000","last_undegraded":"2024-09-05T13:17:13.069084+0000","last_fullsized":"2024-09-05T13:17:13.069084+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38772736,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11560,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11557","reported_seq":11691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.107385+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:13.107385+0000","last_peered":"2024-09-05T13:17:13.107385+0000","last_clean":"2024-09-05T13:17:13.107385+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:13.107385+0000","last_undegraded":"2024-09-05T13:17:13.107385+0000","last_fullsized":"2024-09-05T13:17:13.107385+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8097,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11672","reported_seq":11810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.198812+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:13.198812+0000","last_peered":"2024-09-05T13:17:13.198812+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:13.198812+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8127,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11363","reported_seq":13712,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.290352+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:13.290352+0000","last_peered":"2024-09-05T13:17:13.290352+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:13.290352+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37949440,"num_objects":9265,"num_object_clones":0,"num_object_copies":18530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":9265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7682,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":4521984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5933},{"shards":"0,2","objects":3331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11621","reported_seq":11753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.265694+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:13.265694+0000","last_peered":"2024-09-05T13:17:13.265694+0000","last_clean":"2024-09-05T13:17:13.265694+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:13.265694+0000","last_undegraded":"2024-09-05T13:17:13.265694+0000","last_fullsized":"2024-09-05T13:17:13.265694+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8231,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200578+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200578+0000","last_peered":"2024-09-05T13:16:09.200578+0000","last_clean":"2024-09-05T13:16:09.200578+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200578+0000","last_undegraded":"2024-09-05T13:16:09.200578+0000","last_fullsized":"2024-09-05T13:16:09.200578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11651","reported_seq":11786,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:12.935240+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:12.935240+0000","last_peered":"2024-09-05T13:17:12.935240+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:12.935240+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9305,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4991}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11532","reported_seq":11670,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.241767+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:13.241767+0000","last_peered":"2024-09-05T13:17:13.241767+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:13.241767+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7899,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4924},{"shards":"0,2","objects":3381}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11588","reported_seq":20489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.092195+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:13.092195+0000","last_peered":"2024-09-05T13:17:13.092195+0000","last_clean":"2024-09-05T13:17:13.092195+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:13.092195+0000","last_undegraded":"2024-09-05T13:17:13.092195+0000","last_fullsized":"2024-09-05T13:17:13.092195+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9183,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11347","reported_seq":11470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.040020+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:13.040020+0000","last_peered":"2024-09-05T13:17:13.040020+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:13.040020+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37826560,"num_objects":9235,"num_object_clones":0,"num_object_copies":18470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8856,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3334},{"shards":"2","objects":4848}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11562","reported_seq":11702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.174231+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:13.174231+0000","last_peered":"2024-09-05T13:17:13.174231+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:13.174231+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38158336,"num_objects":9316,"num_object_clones":0,"num_object_copies":18632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4897},{"shards":"0,2","objects":3297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11650","reported_seq":11837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.258888+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:13.258888+0000","last_peered":"2024-09-05T13:17:13.258888+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:13.258888+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38723584,"num_objects":9454,"num_object_clones":0,"num_object_copies":18908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7549,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5010},{"shards":"0,2","objects":3350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11613","reported_seq":11752,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.301069+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:13.301069+0000","last_peered":"2024-09-05T13:17:13.301069+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:13.301069+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7743,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4901},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11706","reported_seq":11844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.220182+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:13.220182+0000","last_peered":"2024-09-05T13:17:13.220182+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:13.220182+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38855139,"num_objects":9487,"num_object_clones":0,"num_object_copies":18974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8374,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4922},{"shards":"0,2","objects":3461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":616006139,"num_objects":150394,"num_object_clones":0,"num_object_copies":300788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":150394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140793,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5496,"num_bytes_recovered":22503424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1299337216,"data_stored":1299245973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736948,"kb_used_data":444260,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93634892,"statfs":{"total":96636764160,"available":95882129408,"internally_reserved":0,"allocated":454922240,"data_stored":454662683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907044,"kb_used_data":524180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93464796,"statfs":{"total":96636764160,"available":95707951104,"internally_reserved":0,"allocated":536760320,"data_stored":536500758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,0,3,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565628,"kb_used_data":303468,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93806212,"statfs":{"total":96636764160,"available":96057561088,"internally_reserved":0,"allocated":310751232,"data_stored":310415338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268431023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":535060480,"data_stored":535056867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310251520,"data_stored":310171575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454025216,"data_stored":454017531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:17.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:17.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11517", 'reported_seq': 11592, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.239814+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:13.239814+0000', 'last_peered': '2024-09-05T13:17:13.239814+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:13.239814+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38727680, 'num_objects': 9455, 'num_object_clones': 0, 'num_object_copies': 18910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11517, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5034}, {'shards': '0,2', 'objects': 3389}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:17.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11739", 'reported_seq': 11868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.083094+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:13.083094+0000', 'last_peered': '2024-09-05T13:17:13.083094+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:13.083094+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38731800, 'num_objects': 9457, 'num_object_clones': 0, 'num_object_copies': 18914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9457, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9550, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3353}, {'shards': '2', 'objects': 4965}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11672", 'reported_seq': 11810, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.198812+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:13.198812+0000', 'last_peered': '2024-09-05T13:17:13.198812+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:13.198812+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38842368, 'num_objects': 9483, 'num_object_clones': 0, 'num_object_copies': 18966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8127, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5032}, {'shards': '0,2', 'objects': 3359}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11363", 'reported_seq': 13712, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.290352+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:13.290352+0000', 'last_peered': '2024-09-05T13:17:13.290352+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:13.290352+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37949440, 'num_objects': 9265, 'num_object_clones': 0, 'num_object_copies': 18530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 3331, 'num_objects_unfound': 0, 'num_objects_dirty': 9265, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7682, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1104, 'num_bytes_recovered': 4521984, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5933}, {'shards': '0,2', 'objects': 3331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11651", 'reported_seq': 11786, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:12.935240+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:12.935240+0000', 'last_peered': '2024-09-05T13:17:12.935240+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:12.935240+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38674432, 'num_objects': 9442, 'num_object_clones': 0, 'num_object_copies': 18884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9305, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3350}, {'shards': '2', 'objects': 4991}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:17.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11532", 'reported_seq': 11670, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.241767+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:13.241767+0000', 'last_peered': '2024-09-05T13:17:13.241767+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:13.241767+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38412288, 'num_objects': 9378, 'num_object_clones': 0, 'num_object_copies': 18756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9378, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7899, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4924}, {'shards': '0,2', 'objects': 3381}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11347", 'reported_seq': 11470, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.040020+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:13.040020+0000', 'last_peered': '2024-09-05T13:17:13.040020+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:13.040020+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37826560, 'num_objects': 9235, 'num_object_clones': 0, 'num_object_copies': 18470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8856, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3334}, {'shards': '2', 'objects': 4848}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11562", 'reported_seq': 11702, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.174231+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:13.174231+0000', 'last_peered': '2024-09-05T13:17:13.174231+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:13.174231+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38158336, 'num_objects': 9316, 'num_object_clones': 0, 'num_object_copies': 18632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7713, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4897}, {'shards': '0,2', 'objects': 3297}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11650", 'reported_seq': 11837, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.258888+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:13.258888+0000', 'last_peered': '2024-09-05T13:17:13.258888+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:13.258888+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38723584, 'num_objects': 9454, 'num_object_clones': 0, 'num_object_copies': 18908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9454, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7549, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5010}, {'shards': '0,2', 'objects': 3350}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11613", 'reported_seq': 11752, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.301069+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:13.301069+0000', 'last_peered': '2024-09-05T13:17:13.301069+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:13.301069+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38776832, 'num_objects': 9467, 'num_object_clones': 0, 'num_object_copies': 18934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7743, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4901}, {'shards': '0,2', 'objects': 3495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11706", 'reported_seq': 11844, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:13.220182+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:13.220182+0000', 'last_peered': '2024-09-05T13:17:13.220182+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:13.220182+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38855139, 'num_objects': 9487, 'num_object_clones': 0, 'num_object_copies': 18974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8374, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4922}, {'shards': '0,2', 'objects': 3461}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:17.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:17.779 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:17.779 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/300792 objects degraded (14.850%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":364,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":150396,"data_bytes":616006158,"bytes_used":2262650880,"bytes_avail":287647641600,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":300792,"degraded_ratio":0.14850461448442778,"misplaced_objects":3331,"misplaced_total":300792,"misplaced_ratio":0.011074097715364771,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59027,"recovering_keys_per_sec":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":361},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:17.808 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:17.808 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:18.373 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:18.375 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:18.399 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-05T13:17:17.619504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616006158,"num_objects":150396,"num_object_clones":0,"num_object_copies":300792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":150396,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140795,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5497,"num_bytes_recovered":22503443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2217200,"kb_used_data":1271168,"kb_used_omap":12,"kb_used_meta":945971,"kb_avail":280898320,"statfs":{"total":289910292480,"available":287639879680,"internally_reserved":0,"allocated":1301676032,"data_stored":1300821019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":968674310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,1,2,6,13,12,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17788928,"num_objects":-4343,"num_object_clones":0,"num_object_copies":-8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":-4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4343,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004692"},"pg_stats":[{"pgid":"2.f","version":"45'11517","reported_seq":11592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.239814+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:13.239814+0000","last_peered":"2024-09-05T13:17:13.239814+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:13.239814+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38727680,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11517,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5034},{"shards":"0,2","objects":3389}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11397","reported_seq":11502,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.086931+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:13.086931+0000","last_peered":"2024-09-05T13:17:13.086931+0000","last_clean":"2024-09-05T13:17:13.086931+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:13.086931+0000","last_undegraded":"2024-09-05T13:17:13.086931+0000","last_fullsized":"2024-09-05T13:17:13.086931+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9407,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11739","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.083094+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:13.083094+0000","last_peered":"2024-09-05T13:17:13.083094+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:13.083094+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731800,"num_objects":9457,"num_object_clones":0,"num_object_copies":18914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9550,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3353},{"shards":"2","objects":4965}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11560","reported_seq":11630,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.069084+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:13.069084+0000","last_peered":"2024-09-05T13:17:13.069084+0000","last_clean":"2024-09-05T13:17:13.069084+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:13.069084+0000","last_undegraded":"2024-09-05T13:17:13.069084+0000","last_fullsized":"2024-09-05T13:17:13.069084+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38772736,"num_objects":9466,"num_object_clones":0,"num_object_copies":18932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11560,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11557","reported_seq":11691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.107385+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:13.107385+0000","last_peered":"2024-09-05T13:17:13.107385+0000","last_clean":"2024-09-05T13:17:13.107385+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:13.107385+0000","last_undegraded":"2024-09-05T13:17:13.107385+0000","last_fullsized":"2024-09-05T13:17:13.107385+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8097,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11672","reported_seq":11810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.198812+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:13.198812+0000","last_peered":"2024-09-05T13:17:13.198812+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:13.198812+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8127,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5032},{"shards":"0,2","objects":3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11363","reported_seq":13712,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.290352+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:13.290352+0000","last_peered":"2024-09-05T13:17:13.290352+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:13.290352+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37949440,"num_objects":9265,"num_object_clones":0,"num_object_copies":18530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":9265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7682,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":4521984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5933},{"shards":"0,2","objects":3331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11621","reported_seq":11753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.265694+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:13.265694+0000","last_peered":"2024-09-05T13:17:13.265694+0000","last_clean":"2024-09-05T13:17:13.265694+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:13.265694+0000","last_undegraded":"2024-09-05T13:17:13.265694+0000","last_fullsized":"2024-09-05T13:17:13.265694+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8231,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11651","reported_seq":11786,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:12.935240+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:12.935240+0000","last_peered":"2024-09-05T13:17:12.935240+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:12.935240+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9305,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3350},{"shards":"2","objects":4991}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11532","reported_seq":11670,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.241767+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:13.241767+0000","last_peered":"2024-09-05T13:17:13.241767+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:13.241767+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38412288,"num_objects":9378,"num_object_clones":0,"num_object_copies":18756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7899,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4924},{"shards":"0,2","objects":3381}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11588","reported_seq":20489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:13.092195+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:13.092195+0000","last_peered":"2024-09-05T13:17:13.092195+0000","last_clean":"2024-09-05T13:17:13.092195+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:13.092195+0000","last_undegraded":"2024-09-05T13:17:13.092195+0000","last_fullsized":"2024-09-05T13:17:13.092195+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9183,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11347","reported_seq":11470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.040020+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:13.040020+0000","last_peered":"2024-09-05T13:17:13.040020+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:13.040020+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37826560,"num_objects":9235,"num_object_clones":0,"num_object_copies":18470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8856,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3334},{"shards":"2","objects":4848}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11562","reported_seq":11702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.174231+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:13.174231+0000","last_peered":"2024-09-05T13:17:13.174231+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:13.174231+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38158336,"num_objects":9316,"num_object_clones":0,"num_object_copies":18632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7713,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4897},{"shards":"0,2","objects":3297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11650","reported_seq":11837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.258888+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:13.258888+0000","last_peered":"2024-09-05T13:17:13.258888+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:13.258888+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38723584,"num_objects":9454,"num_object_clones":0,"num_object_copies":18908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7549,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5010},{"shards":"0,2","objects":3350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11613","reported_seq":11752,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.301069+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:13.301069+0000","last_peered":"2024-09-05T13:17:13.301069+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:13.301069+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7743,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4901},{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11706","reported_seq":11844,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:13.220182+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:13.220182+0000","last_peered":"2024-09-05T13:17:13.220182+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:13.220182+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38855139,"num_objects":9487,"num_object_clones":0,"num_object_copies":18974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8374,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4922},{"shards":"0,2","objects":3461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":616006139,"num_objects":150394,"num_object_clones":0,"num_object_copies":300788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3331,"num_objects_unfound":0,"num_objects_dirty":150394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140793,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5496,"num_bytes_recovered":22503424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1298599936,"data_stored":1298508693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736948,"kb_used_data":444260,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93634892,"statfs":{"total":96636764160,"available":95882129408,"internally_reserved":0,"allocated":454922240,"data_stored":454662683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,5,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":907044,"kb_used_data":524180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93464796,"statfs":{"total":96636764160,"available":95707951104,"internally_reserved":0,"allocated":536760320,"data_stored":536500758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,0,3,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573208,"kb_used_data":302728,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93798632,"statfs":{"total":96636764160,"available":96049799168,"internally_reserved":0,"allocated":309993472,"data_stored":309657578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276950703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":535060480,"data_stored":535056867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309514240,"data_stored":309434295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454025216,"data_stored":454017531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:21.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:21.973 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:21.994 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-05T13:17:21.621036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607044110,"num_objects":148208,"num_object_clones":0,"num_object_copies":296416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3246,"num_objects_unfound":0,"num_objects_dirty":148208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142983,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5582,"num_bytes_recovered":22851603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2184732,"kb_used_data":1259484,"kb_used_omap":12,"kb_used_meta":925171,"kb_avail":280930788,"statfs":{"total":289910292480,"available":287673126912,"internally_reserved":0,"allocated":1289711616,"data_stored":1288856603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":947375110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,8,10,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17809408,"num_objects":-4348,"num_object_clones":0,"num_object_copies":-8696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-172,"num_objects_unfound":0,"num_objects_dirty":-4348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4348,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004989"},"pg_stats":[{"pgid":"2.f","version":"45'11664","reported_seq":11738,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.222187+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:18.222187+0000","last_peered":"2024-09-05T13:17:18.222187+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:18.222187+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38125568,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11664,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":3242}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11551","reported_seq":11656,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.110533+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:18.110533+0000","last_peered":"2024-09-05T13:17:18.110533+0000","last_clean":"2024-09-05T13:17:18.110533+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:18.110533+0000","last_undegraded":"2024-09-05T13:17:18.110533+0000","last_fullsized":"2024-09-05T13:17:18.110533+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37695488,"num_objects":9203,"num_object_clones":0,"num_object_copies":18406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9561,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11867","reported_seq":11996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.135088+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:18.135088+0000","last_peered":"2024-09-05T13:17:18.135088+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:18.135088+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38207512,"num_objects":9329,"num_object_clones":0,"num_object_copies":18658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9678,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3225},{"shards":"2","objects":4837}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11685","reported_seq":11755,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.051769+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:18.051769+0000","last_peered":"2024-09-05T13:17:18.051769+0000","last_clean":"2024-09-05T13:17:18.051769+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:18.051769+0000","last_undegraded":"2024-09-05T13:17:18.051769+0000","last_fullsized":"2024-09-05T13:17:18.051769+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38260736,"num_objects":9341,"num_object_clones":0,"num_object_copies":18682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11685,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11699","reported_seq":11833,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.187406+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:18.187406+0000","last_peered":"2024-09-05T13:17:18.187406+0000","last_clean":"2024-09-05T13:17:18.187406+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:18.187406+0000","last_undegraded":"2024-09-05T13:17:18.187406+0000","last_fullsized":"2024-09-05T13:17:18.187406+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11807","reported_seq":11945,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.243754+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:18.243754+0000","last_peered":"2024-09-05T13:17:18.243754+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:18.243754+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8262,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4897},{"shards":"0,2","objects":3224}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11501","reported_seq":14019,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.278209+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:18.278209+0000","last_peered":"2024-09-05T13:17:18.278209+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:18.278209+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3246,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7820,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1189,"num_bytes_recovered":4870144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5880},{"shards":"0,2","objects":3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11774","reported_seq":11906,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.323036+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:18.323036+0000","last_peered":"2024-09-05T13:17:18.323036+0000","last_clean":"2024-09-05T13:17:18.323036+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:18.323036+0000","last_undegraded":"2024-09-05T13:17:18.323036+0000","last_fullsized":"2024-09-05T13:17:18.323036+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38215680,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11809","reported_seq":11944,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.071676+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:18.071676+0000","last_peered":"2024-09-05T13:17:18.071676+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:18.071676+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38027264,"num_objects":9284,"num_object_clones":0,"num_object_copies":18568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9463,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3192},{"shards":"2","objects":4833}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11651","reported_seq":11789,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.023206+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:18.023206+0000","last_peered":"2024-09-05T13:17:18.023206+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:18.023206+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8018,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4805},{"shards":"0,2","objects":3262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11710","reported_seq":20611,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.111583+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:18.111583+0000","last_peered":"2024-09-05T13:17:18.111583+0000","last_clean":"2024-09-05T13:17:18.111583+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:18.111583+0000","last_undegraded":"2024-09-05T13:17:18.111583+0000","last_fullsized":"2024-09-05T13:17:18.111583+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37687296,"num_objects":9201,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9305,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11485","reported_seq":11608,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.015441+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:18.015441+0000","last_peered":"2024-09-05T13:17:18.015441+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:18.015441+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261312,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8994,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3196},{"shards":"2","objects":4710}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11692","reported_seq":11832,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.088944+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:18.088944+0000","last_peered":"2024-09-05T13:17:18.088944+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:18.088944+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37625856,"num_objects":9186,"num_object_clones":0,"num_object_copies":18372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7843,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4767},{"shards":"0,2","objects":3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11774","reported_seq":11961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.224465+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:18.224465+0000","last_peered":"2024-09-05T13:17:18.224465+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:18.224465+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38215680,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7673,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4886},{"shards":"0,2","objects":3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11767","reported_seq":11906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.255199+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:18.255199+0000","last_peered":"2024-09-05T13:17:18.255199+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:18.255199+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146048,"num_objects":9313,"num_object_clones":0,"num_object_copies":18626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7897,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4747},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11827","reported_seq":11967,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.273804+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:18.273804+0000","last_peered":"2024-09-05T13:17:18.273804+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:18.273804+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38359523,"num_objects":9366,"num_object_clones":0,"num_object_copies":18732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8495,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4801},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":607044091,"num_objects":148206,"num_object_clones":0,"num_object_copies":296412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3246,"num_objects_unfound":0,"num_objects_dirty":148206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142981,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5581,"num_bytes_recovered":22851584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1286684672,"data_stored":1286593429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720076,"kb_used_data":439860,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93651764,"statfs":{"total":96636764160,"available":95899406336,"internally_reserved":0,"allocated":450416640,"data_stored":450157083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887904,"kb_used_data":517512,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93483936,"statfs":{"total":96636764160,"available":95727550464,"internally_reserved":0,"allocated":529932288,"data_stored":529672726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576752,"kb_used_data":302112,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93795088,"statfs":{"total":96636764160,"available":96046170112,"internally_reserved":0,"allocated":309362688,"data_stored":309026794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":528351232,"data_stored":528347619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308879360,"data_stored":308799415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":449454080,"data_stored":449446395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:21.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:22.570 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:22.606 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-05T13:17:21.621036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607044110,"num_objects":148208,"num_object_clones":0,"num_object_copies":296416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3246,"num_objects_unfound":0,"num_objects_dirty":148208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142983,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5582,"num_bytes_recovered":22851603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2184732,"kb_used_data":1259484,"kb_used_omap":12,"kb_used_meta":925171,"kb_avail":280930788,"statfs":{"total":289910292480,"available":287673126912,"internally_reserved":0,"allocated":1289711616,"data_stored":1288856603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":947375110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,8,10,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17809408,"num_objects":-4348,"num_object_clones":0,"num_object_copies":-8696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-172,"num_objects_unfound":0,"num_objects_dirty":-4348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4348,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004989"},"pg_stats":[{"pgid":"2.f","version":"45'11664","reported_seq":11738,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.222187+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:18.222187+0000","last_peered":"2024-09-05T13:17:18.222187+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:18.222187+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38125568,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11664,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4887},{"shards":"0,2","objects":3242}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11551","reported_seq":11656,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.110533+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:18.110533+0000","last_peered":"2024-09-05T13:17:18.110533+0000","last_clean":"2024-09-05T13:17:18.110533+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:18.110533+0000","last_undegraded":"2024-09-05T13:17:18.110533+0000","last_fullsized":"2024-09-05T13:17:18.110533+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37695488,"num_objects":9203,"num_object_clones":0,"num_object_copies":18406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9561,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11867","reported_seq":11996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.135088+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:18.135088+0000","last_peered":"2024-09-05T13:17:18.135088+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:18.135088+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38207512,"num_objects":9329,"num_object_clones":0,"num_object_copies":18658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9678,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3225},{"shards":"2","objects":4837}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11685","reported_seq":11755,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.051769+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:18.051769+0000","last_peered":"2024-09-05T13:17:18.051769+0000","last_clean":"2024-09-05T13:17:18.051769+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:18.051769+0000","last_undegraded":"2024-09-05T13:17:18.051769+0000","last_fullsized":"2024-09-05T13:17:18.051769+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38260736,"num_objects":9341,"num_object_clones":0,"num_object_copies":18682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11685,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11699","reported_seq":11833,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.187406+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:18.187406+0000","last_peered":"2024-09-05T13:17:18.187406+0000","last_clean":"2024-09-05T13:17:18.187406+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:18.187406+0000","last_undegraded":"2024-09-05T13:17:18.187406+0000","last_fullsized":"2024-09-05T13:17:18.187406+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11807","reported_seq":11945,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.243754+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:18.243754+0000","last_peered":"2024-09-05T13:17:18.243754+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:18.243754+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8262,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4897},{"shards":"0,2","objects":3224}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11501","reported_seq":14019,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.278209+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:18.278209+0000","last_peered":"2024-09-05T13:17:18.278209+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:18.278209+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":3246,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7820,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1189,"num_bytes_recovered":4870144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5880},{"shards":"0,2","objects":3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11774","reported_seq":11906,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.323036+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:18.323036+0000","last_peered":"2024-09-05T13:17:18.323036+0000","last_clean":"2024-09-05T13:17:18.323036+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:18.323036+0000","last_undegraded":"2024-09-05T13:17:18.323036+0000","last_fullsized":"2024-09-05T13:17:18.323036+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38215680,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8384,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11809","reported_seq":11944,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.071676+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:18.071676+0000","last_peered":"2024-09-05T13:17:18.071676+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:18.071676+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38027264,"num_objects":9284,"num_object_clones":0,"num_object_copies":18568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9463,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3192},{"shards":"2","objects":4833}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11651","reported_seq":11789,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.023206+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:18.023206+0000","last_peered":"2024-09-05T13:17:18.023206+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:18.023206+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8018,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4805},{"shards":"0,2","objects":3262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11710","reported_seq":20611,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:18.111583+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:18.111583+0000","last_peered":"2024-09-05T13:17:18.111583+0000","last_clean":"2024-09-05T13:17:18.111583+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:18.111583+0000","last_undegraded":"2024-09-05T13:17:18.111583+0000","last_fullsized":"2024-09-05T13:17:18.111583+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37687296,"num_objects":9201,"num_object_clones":0,"num_object_copies":18402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9305,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11485","reported_seq":11608,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.015441+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:18.015441+0000","last_peered":"2024-09-05T13:17:18.015441+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:18.015441+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261312,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8994,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3196},{"shards":"2","objects":4710}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11692","reported_seq":11832,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.088944+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:18.088944+0000","last_peered":"2024-09-05T13:17:18.088944+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:18.088944+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37625856,"num_objects":9186,"num_object_clones":0,"num_object_copies":18372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7843,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4767},{"shards":"0,2","objects":3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11774","reported_seq":11961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.224465+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:18.224465+0000","last_peered":"2024-09-05T13:17:18.224465+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:18.224465+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38215680,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7673,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4886},{"shards":"0,2","objects":3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11767","reported_seq":11906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.255199+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:18.255199+0000","last_peered":"2024-09-05T13:17:18.255199+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:18.255199+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146048,"num_objects":9313,"num_object_clones":0,"num_object_copies":18626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7897,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4747},{"shards":"0,2","objects":3341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11827","reported_seq":11967,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:18.273804+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:18.273804+0000","last_peered":"2024-09-05T13:17:18.273804+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:18.273804+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38359523,"num_objects":9366,"num_object_clones":0,"num_object_copies":18732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8495,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4801},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":607044091,"num_objects":148206,"num_object_clones":0,"num_object_copies":296412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44669,"num_objects_misplaced":3246,"num_objects_unfound":0,"num_objects_dirty":148206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142981,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5581,"num_bytes_recovered":22851584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1286684672,"data_stored":1286593429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720076,"kb_used_data":439860,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93651764,"statfs":{"total":96636764160,"available":95899406336,"internally_reserved":0,"allocated":450416640,"data_stored":450157083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887904,"kb_used_data":517512,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93483936,"statfs":{"total":96636764160,"available":95727550464,"internally_reserved":0,"allocated":529932288,"data_stored":529672726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576752,"kb_used_data":302112,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93795088,"statfs":{"total":96636764160,"available":96046170112,"internally_reserved":0,"allocated":309362688,"data_stored":309026794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":528351232,"data_stored":528347619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308879360,"data_stored":308799415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":449454080,"data_stored":449446395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:22.607 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:22.607 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11664", 'reported_seq': 11738, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.222187+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:18.222187+0000', 'last_peered': '2024-09-05T13:17:18.222187+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:18.222187+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38125568, 'num_objects': 9308, 'num_object_clones': 0, 'num_object_copies': 18616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11664, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4887}, {'shards': '0,2', 'objects': 3242}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11867", 'reported_seq': 11996, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.135088+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:18.135088+0000', 'last_peered': '2024-09-05T13:17:18.135088+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:18.135088+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38207512, 'num_objects': 9329, 'num_object_clones': 0, 'num_object_copies': 18658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9329, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9678, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3225}, {'shards': '2', 'objects': 4837}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11807", 'reported_seq': 11945, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.243754+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:18.243754+0000', 'last_peered': '2024-09-05T13:17:18.243754+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:18.243754+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38289408, 'num_objects': 9348, 'num_object_clones': 0, 'num_object_copies': 18696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9348, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8262, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4897}, {'shards': '0,2', 'objects': 3224}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11501", 'reported_seq': 14019, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.278209+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:18.278209+0000', 'last_peered': '2024-09-05T13:17:18.278209+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:18.278209+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37384192, 'num_objects': 9127, 'num_object_clones': 0, 'num_object_copies': 18254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 3246, 'num_objects_unfound': 0, 'num_objects_dirty': 9127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7820, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1189, 'num_bytes_recovered': 4870144, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5880}, {'shards': '0,2', 'objects': 3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11809", 'reported_seq': 11944, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.071676+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:18.071676+0000', 'last_peered': '2024-09-05T13:17:18.071676+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:18.071676+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38027264, 'num_objects': 9284, 'num_object_clones': 0, 'num_object_copies': 18568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9284, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9463, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3192}, {'shards': '2', 'objects': 4833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:22.608 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11651", 'reported_seq': 11789, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.023206+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:18.023206+0000', 'last_peered': '2024-09-05T13:17:18.023206+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:18.023206+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37924864, 'num_objects': 9259, 'num_object_clones': 0, 'num_object_copies': 18518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8018, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4805}, {'shards': '0,2', 'objects': 3262}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11485", 'reported_seq': 11608, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.015441+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:18.015441+0000', 'last_peered': '2024-09-05T13:17:18.015441+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:18.015441+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37261312, 'num_objects': 9097, 'num_object_clones': 0, 'num_object_copies': 18194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8994, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3196}, {'shards': '2', 'objects': 4710}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11692", 'reported_seq': 11832, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.088944+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:18.088944+0000', 'last_peered': '2024-09-05T13:17:18.088944+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:18.088944+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37625856, 'num_objects': 9186, 'num_object_clones': 0, 'num_object_copies': 18372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7843, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4767}, {'shards': '0,2', 'objects': 3167}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11774", 'reported_seq': 11961, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.224465+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:18.224465+0000', 'last_peered': '2024-09-05T13:17:18.224465+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:18.224465+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38215680, 'num_objects': 9330, 'num_object_clones': 0, 'num_object_copies': 18660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9330, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7673, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4886}, {'shards': '0,2', 'objects': 3226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11767", 'reported_seq': 11906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.255199+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:18.255199+0000', 'last_peered': '2024-09-05T13:17:18.255199+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:18.255199+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38146048, 'num_objects': 9313, 'num_object_clones': 0, 'num_object_copies': 18626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7897, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4747}, {'shards': '0,2', 'objects': 3341}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:22.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11827", 'reported_seq': 11967, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:18.273804+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:18.273804+0000', 'last_peered': '2024-09-05T13:17:18.273804+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:18.273804+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38359523, 'num_objects': 9366, 'num_object_clones': 0, 'num_object_copies': 18732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8495, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4801}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:22.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:23.318 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:23.318 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44669/296416 objects degraded (15.070%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":370,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":148208,"data_bytes":607044110,"bytes_used":2237165568,"bytes_avail":287673126912,"bytes_total":289910292480,"degraded_objects":44669,"degraded_total":296416,"degraded_ratio":0.15069699341466047,"misplaced_objects":3246,"misplaced_total":296416,"misplaced_ratio":0.010950825866349994,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":58684,"recovering_keys_per_sec":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":362},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:23.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:23.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:23.897 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:23.922 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-05T13:17:23.622509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597901838,"num_objects":145976,"num_object_clones":0,"num_object_copies":291952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44709,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":145976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145215,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5669,"num_bytes_recovered":23207955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2181928,"kb_used_data":1248368,"kb_used_omap":12,"kb_used_meta":933491,"kb_avail":280933592,"statfs":{"total":289910292480,"available":287675998208,"internally_reserved":0,"allocated":1278328832,"data_stored":1277473819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":955894790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,1,9,9,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26951680,"num_objects":-6580,"num_object_clones":0,"num_object_copies":-13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":-6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6580,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004709"},"pg_stats":[{"pgid":"2.f","version":"45'11807","reported_seq":11880,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.370665+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:23.370665+0000","last_peered":"2024-09-05T13:17:23.370665+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:23.370665+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11807,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4744},{"shards":"0,2","objects":3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11687","reported_seq":11792,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.115660+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:23.115660+0000","last_peered":"2024-09-05T13:17:23.115660+0000","last_clean":"2024-09-05T13:17:23.115660+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:23.115660+0000","last_undegraded":"2024-09-05T13:17:23.115660+0000","last_fullsized":"2024-09-05T13:17:23.115660+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9697,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11997","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.145255+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:23.145255+0000","last_peered":"2024-09-05T13:17:23.145255+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:23.145255+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37675032,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9808,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3095},{"shards":"2","objects":4707}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11814","reported_seq":11883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.144962+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:23.144962+0000","last_peered":"2024-09-05T13:17:23.144962+0000","last_clean":"2024-09-05T13:17:23.144962+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:23.144962+0000","last_undegraded":"2024-09-05T13:17:23.144962+0000","last_fullsized":"2024-09-05T13:17:23.144962+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732352,"num_objects":9212,"num_object_clones":0,"num_object_copies":18424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11838","reported_seq":11971,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.348625+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:23.348625+0000","last_peered":"2024-09-05T13:17:23.348625+0000","last_clean":"2024-09-05T13:17:23.348625+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:23.348625+0000","last_undegraded":"2024-09-05T13:17:23.348625+0000","last_fullsized":"2024-09-05T13:17:23.348625+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11932","reported_seq":12070,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.279113+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:23.279113+0000","last_peered":"2024-09-05T13:17:23.279113+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:23.279113+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777408,"num_objects":9223,"num_object_clones":0,"num_object_copies":18446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8387,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4772},{"shards":"0,2","objects":3099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11632","reported_seq":14325,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.342122+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:23.342122+0000","last_peered":"2024-09-05T13:17:23.342122+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:23.342122+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36847616,"num_objects":8996,"num_object_clones":0,"num_object_copies":17992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":8996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7951,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1276,"num_bytes_recovered":5226496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11924","reported_seq":12056,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.112292+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:23.112292+0000","last_peered":"2024-09-05T13:17:23.112292+0000","last_clean":"2024-09-05T13:17:23.112292+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:23.112292+0000","last_undegraded":"2024-09-05T13:17:23.112292+0000","last_fullsized":"2024-09-05T13:17:23.112292+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8534,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11949","reported_seq":12084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.011251+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:23.011251+0000","last_peered":"2024-09-05T13:17:23.011251+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:23.011251+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9603,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3052},{"shards":"2","objects":4693}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11793","reported_seq":11930,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.307730+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:23.307730+0000","last_peered":"2024-09-05T13:17:23.307730+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:23.307730+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8160,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":3120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11848","reported_seq":20749,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:22.978947+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:22.978947+0000","last_peered":"2024-09-05T13:17:22.978947+0000","last_clean":"2024-09-05T13:17:22.978947+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:22.978947+0000","last_undegraded":"2024-09-05T13:17:22.978947+0000","last_fullsized":"2024-09-05T13:17:22.978947+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122048,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9443,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11619","reported_seq":11742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.097426+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:23.097426+0000","last_peered":"2024-09-05T13:17:23.097426+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:23.097426+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3062},{"shards":"2","objects":4576}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11830","reported_seq":11970,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.305976+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:23.305976+0000","last_peered":"2024-09-05T13:17:23.305976+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:23.305976+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37060608,"num_objects":9048,"num_object_clones":0,"num_object_copies":18096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7981,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11939","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.350606+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:23.350606+0000","last_peered":"2024-09-05T13:17:23.350606+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:23.350606+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7838,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4721},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11926","reported_seq":12064,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.385115+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:23.385115+0000","last_peered":"2024-09-05T13:17:23.385115+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:23.385115+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8056,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11960","reported_seq":12100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.229478+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:23.229478+0000","last_peered":"2024-09-05T13:17:23.229478+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:23.229478+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37814755,"num_objects":9233,"num_object_clones":0,"num_object_copies":18466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4668},{"shards":"0,2","objects":3207}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":597901819,"num_objects":145974,"num_object_clones":0,"num_object_copies":291948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44709,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":145974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145213,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5668,"num_bytes_recovered":23207936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1275265024,"data_stored":1275173781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719856,"kb_used_data":435488,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93651984,"statfs":{"total":96636764160,"available":95899631616,"internally_reserved":0,"allocated":445939712,"data_stored":445680155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885320,"kb_used_data":510768,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93486520,"statfs":{"total":96636764160,"available":95730196480,"internally_reserved":0,"allocated":523026432,"data_stored":522766870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576752,"kb_used_data":302112,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93795088,"statfs":{"total":96636764160,"available":96046170112,"internally_reserved":0,"allocated":309362688,"data_stored":309026794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521412608,"data_stored":521408995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308879360,"data_stored":308799415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444973056,"data_stored":444965371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:26.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:27.511 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:27.512 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:27.535 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-05T13:17:25.623369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597901838,"num_objects":145976,"num_object_clones":0,"num_object_copies":291952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44709,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":145976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145215,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5669,"num_bytes_recovered":23207955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2181928,"kb_used_data":1248368,"kb_used_omap":12,"kb_used_meta":933491,"kb_avail":280933592,"statfs":{"total":289910292480,"available":287675998208,"internally_reserved":0,"allocated":1278328832,"data_stored":1277473819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":955894790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,1,9,9,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18104320,"num_objects":-4420,"num_object_clones":0,"num_object_copies":-8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":-212,"num_objects_unfound":0,"num_objects_dirty":-4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005098"},"pg_stats":[{"pgid":"2.f","version":"45'11807","reported_seq":11880,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.370665+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:23.370665+0000","last_peered":"2024-09-05T13:17:23.370665+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:23.370665+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11807,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4744},{"shards":"0,2","objects":3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11687","reported_seq":11792,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.115660+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:23.115660+0000","last_peered":"2024-09-05T13:17:23.115660+0000","last_clean":"2024-09-05T13:17:23.115660+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:23.115660+0000","last_undegraded":"2024-09-05T13:17:23.115660+0000","last_fullsized":"2024-09-05T13:17:23.115660+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9697,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11997","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.145255+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:23.145255+0000","last_peered":"2024-09-05T13:17:23.145255+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:23.145255+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37675032,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9808,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3095},{"shards":"2","objects":4707}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11814","reported_seq":11883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.144962+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:23.144962+0000","last_peered":"2024-09-05T13:17:23.144962+0000","last_clean":"2024-09-05T13:17:23.144962+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:23.144962+0000","last_undegraded":"2024-09-05T13:17:23.144962+0000","last_fullsized":"2024-09-05T13:17:23.144962+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732352,"num_objects":9212,"num_object_clones":0,"num_object_copies":18424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11838","reported_seq":11971,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.348625+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:23.348625+0000","last_peered":"2024-09-05T13:17:23.348625+0000","last_clean":"2024-09-05T13:17:23.348625+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:23.348625+0000","last_undegraded":"2024-09-05T13:17:23.348625+0000","last_fullsized":"2024-09-05T13:17:23.348625+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11932","reported_seq":12070,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.279113+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:23.279113+0000","last_peered":"2024-09-05T13:17:23.279113+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:23.279113+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777408,"num_objects":9223,"num_object_clones":0,"num_object_copies":18446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8387,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4772},{"shards":"0,2","objects":3099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11632","reported_seq":14325,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.342122+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:23.342122+0000","last_peered":"2024-09-05T13:17:23.342122+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:23.342122+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36847616,"num_objects":8996,"num_object_clones":0,"num_object_copies":17992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":8996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7951,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1276,"num_bytes_recovered":5226496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11924","reported_seq":12056,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.112292+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:23.112292+0000","last_peered":"2024-09-05T13:17:23.112292+0000","last_clean":"2024-09-05T13:17:23.112292+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:23.112292+0000","last_undegraded":"2024-09-05T13:17:23.112292+0000","last_fullsized":"2024-09-05T13:17:23.112292+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8534,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11949","reported_seq":12084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.011251+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:23.011251+0000","last_peered":"2024-09-05T13:17:23.011251+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:23.011251+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9603,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3052},{"shards":"2","objects":4693}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11793","reported_seq":11930,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.307730+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:23.307730+0000","last_peered":"2024-09-05T13:17:23.307730+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:23.307730+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8160,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":3120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11848","reported_seq":20749,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:22.978947+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:22.978947+0000","last_peered":"2024-09-05T13:17:22.978947+0000","last_clean":"2024-09-05T13:17:22.978947+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:22.978947+0000","last_undegraded":"2024-09-05T13:17:22.978947+0000","last_fullsized":"2024-09-05T13:17:22.978947+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122048,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9443,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200942+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200942+0000","last_peered":"2024-09-05T13:16:09.200942+0000","last_clean":"2024-09-05T13:16:09.200942+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200942+0000","last_undegraded":"2024-09-05T13:16:09.200942+0000","last_fullsized":"2024-09-05T13:16:09.200942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11619","reported_seq":11742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.097426+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:23.097426+0000","last_peered":"2024-09-05T13:17:23.097426+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:23.097426+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3062},{"shards":"2","objects":4576}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11830","reported_seq":11970,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.305976+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:23.305976+0000","last_peered":"2024-09-05T13:17:23.305976+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:23.305976+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37060608,"num_objects":9048,"num_object_clones":0,"num_object_copies":18096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7981,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200871+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200871+0000","last_peered":"2024-09-05T13:16:09.200871+0000","last_clean":"2024-09-05T13:16:09.200871+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200871+0000","last_undegraded":"2024-09-05T13:16:09.200871+0000","last_fullsized":"2024-09-05T13:16:09.200871+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11939","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.350606+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:23.350606+0000","last_peered":"2024-09-05T13:17:23.350606+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:23.350606+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7838,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4721},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11926","reported_seq":12064,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.385115+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:23.385115+0000","last_peered":"2024-09-05T13:17:23.385115+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:23.385115+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8056,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201450+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201450+0000","last_peered":"2024-09-05T13:16:09.201450+0000","last_clean":"2024-09-05T13:16:09.201450+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201450+0000","last_undegraded":"2024-09-05T13:16:09.201450+0000","last_fullsized":"2024-09-05T13:16:09.201450+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11960","reported_seq":12100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.229478+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:23.229478+0000","last_peered":"2024-09-05T13:17:23.229478+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:23.229478+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37814755,"num_objects":9233,"num_object_clones":0,"num_object_copies":18466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4668},{"shards":"0,2","objects":3207}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":597901819,"num_objects":145974,"num_object_clones":0,"num_object_copies":291948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44709,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":145974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145213,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5668,"num_bytes_recovered":23207936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1275265024,"data_stored":1275173781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719856,"kb_used_data":435488,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93651984,"statfs":{"total":96636764160,"available":95899631616,"internally_reserved":0,"allocated":445939712,"data_stored":445680155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885320,"kb_used_data":510768,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93486520,"statfs":{"total":96636764160,"available":95730196480,"internally_reserved":0,"allocated":523026432,"data_stored":522766870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576752,"kb_used_data":302112,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93795088,"statfs":{"total":96636764160,"available":96046170112,"internally_reserved":0,"allocated":309362688,"data_stored":309026794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281210543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521412608,"data_stored":521408995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308879360,"data_stored":308799415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444973056,"data_stored":444965371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:27.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:28.099 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:28.101 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:28.135 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-05T13:17:27.624159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597901838,"num_objects":145976,"num_object_clones":0,"num_object_copies":291952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44709,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":145976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145215,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5669,"num_bytes_recovered":23207955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2190976,"kb_used_data":1247688,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":280924544,"statfs":{"total":289910292480,"available":287666733056,"internally_reserved":0,"allocated":1277632512,"data_stored":1276777499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":965856262},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,2,8,8,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18104320,"num_objects":-4420,"num_object_clones":0,"num_object_copies":-8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":-212,"num_objects_unfound":0,"num_objects_dirty":-4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005271"},"pg_stats":[{"pgid":"2.f","version":"45'11807","reported_seq":11880,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.370665+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:23.370665+0000","last_peered":"2024-09-05T13:17:23.370665+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:23.370665+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11807,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4744},{"shards":"0,2","objects":3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11687","reported_seq":11792,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.115660+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:23.115660+0000","last_peered":"2024-09-05T13:17:23.115660+0000","last_clean":"2024-09-05T13:17:23.115660+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:23.115660+0000","last_undegraded":"2024-09-05T13:17:23.115660+0000","last_fullsized":"2024-09-05T13:17:23.115660+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9697,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11997","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.145255+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:23.145255+0000","last_peered":"2024-09-05T13:17:23.145255+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:23.145255+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37675032,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9808,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3095},{"shards":"2","objects":4707}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11814","reported_seq":11883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.144962+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:23.144962+0000","last_peered":"2024-09-05T13:17:23.144962+0000","last_clean":"2024-09-05T13:17:23.144962+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:23.144962+0000","last_undegraded":"2024-09-05T13:17:23.144962+0000","last_fullsized":"2024-09-05T13:17:23.144962+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732352,"num_objects":9212,"num_object_clones":0,"num_object_copies":18424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11838","reported_seq":11971,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.348625+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:23.348625+0000","last_peered":"2024-09-05T13:17:23.348625+0000","last_clean":"2024-09-05T13:17:23.348625+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:23.348625+0000","last_undegraded":"2024-09-05T13:17:23.348625+0000","last_fullsized":"2024-09-05T13:17:23.348625+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11932","reported_seq":12070,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.279113+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:23.279113+0000","last_peered":"2024-09-05T13:17:23.279113+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:23.279113+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777408,"num_objects":9223,"num_object_clones":0,"num_object_copies":18446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8387,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4772},{"shards":"0,2","objects":3099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11632","reported_seq":14325,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.342122+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:23.342122+0000","last_peered":"2024-09-05T13:17:23.342122+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:23.342122+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36847616,"num_objects":8996,"num_object_clones":0,"num_object_copies":17992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":8996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7951,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1276,"num_bytes_recovered":5226496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11924","reported_seq":12056,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.112292+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:23.112292+0000","last_peered":"2024-09-05T13:17:23.112292+0000","last_clean":"2024-09-05T13:17:23.112292+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:23.112292+0000","last_undegraded":"2024-09-05T13:17:23.112292+0000","last_fullsized":"2024-09-05T13:17:23.112292+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8534,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11949","reported_seq":12084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.011251+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:23.011251+0000","last_peered":"2024-09-05T13:17:23.011251+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:23.011251+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9603,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3052},{"shards":"2","objects":4693}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11793","reported_seq":11930,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.307730+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:23.307730+0000","last_peered":"2024-09-05T13:17:23.307730+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:23.307730+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8160,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":3120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11848","reported_seq":20749,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:22.978947+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:22.978947+0000","last_peered":"2024-09-05T13:17:22.978947+0000","last_clean":"2024-09-05T13:17:22.978947+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:22.978947+0000","last_undegraded":"2024-09-05T13:17:22.978947+0000","last_fullsized":"2024-09-05T13:17:22.978947+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122048,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9443,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11619","reported_seq":11742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.097426+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:23.097426+0000","last_peered":"2024-09-05T13:17:23.097426+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:23.097426+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3062},{"shards":"2","objects":4576}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11830","reported_seq":11970,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.305976+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:23.305976+0000","last_peered":"2024-09-05T13:17:23.305976+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:23.305976+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37060608,"num_objects":9048,"num_object_clones":0,"num_object_copies":18096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7981,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11939","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.350606+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:23.350606+0000","last_peered":"2024-09-05T13:17:23.350606+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:23.350606+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7838,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4721},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11926","reported_seq":12064,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.385115+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:23.385115+0000","last_peered":"2024-09-05T13:17:23.385115+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:23.385115+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8056,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11960","reported_seq":12100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.229478+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:23.229478+0000","last_peered":"2024-09-05T13:17:23.229478+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:23.229478+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37814755,"num_objects":9233,"num_object_clones":0,"num_object_copies":18466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4668},{"shards":"0,2","objects":3207}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":597901819,"num_objects":145974,"num_object_clones":0,"num_object_copies":291948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44709,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":145974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145213,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5668,"num_bytes_recovered":23207936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1274671104,"data_stored":1274579861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719856,"kb_used_data":435488,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93651984,"statfs":{"total":96636764160,"available":95899631616,"internally_reserved":0,"allocated":445939712,"data_stored":445680155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885320,"kb_used_data":510768,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93486520,"statfs":{"total":96636764160,"available":95730196480,"internally_reserved":0,"allocated":523026432,"data_stored":522766870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585800,"kb_used_data":301432,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93786040,"statfs":{"total":96636764160,"available":96036904960,"internally_reserved":0,"allocated":308666368,"data_stored":308330474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521412608,"data_stored":521408995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308285440,"data_stored":308205495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444973056,"data_stored":444965371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:28.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:28.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11807", 'reported_seq': 11880, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.370665+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:23.370665+0000', 'last_peered': '2024-09-05T13:17:23.370665+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:23.370665+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37539840, 'num_objects': 9165, 'num_object_clones': 0, 'num_object_copies': 18330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11807, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4744}, {'shards': '0,2', 'objects': 3099}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11997", 'reported_seq': 12126, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.145255+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:23.145255+0000', 'last_peered': '2024-09-05T13:17:23.145255+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:23.145255+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37675032, 'num_objects': 9199, 'num_object_clones': 0, 'num_object_copies': 18398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9199, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9808, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3095}, {'shards': '2', 'objects': 4707}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11932", 'reported_seq': 12070, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.279113+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:23.279113+0000', 'last_peered': '2024-09-05T13:17:23.279113+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:23.279113+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37777408, 'num_objects': 9223, 'num_object_clones': 0, 'num_object_copies': 18446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9223, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8387, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4772}, {'shards': '0,2', 'objects': 3099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11632", 'reported_seq': 14325, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.342122+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:23.342122+0000', 'last_peered': '2024-09-05T13:17:23.342122+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:23.342122+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36847616, 'num_objects': 8996, 'num_object_clones': 0, 'num_object_copies': 17992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 41, 'num_objects_misplaced': 3119, 'num_objects_unfound': 0, 'num_objects_dirty': 8996, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7951, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1276, 'num_bytes_recovered': 5226496, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5836}, {'shards': '0,2', 'objects': 3119}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11949", 'reported_seq': 12084, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.011251+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:23.011251+0000', 'last_peered': '2024-09-05T13:17:23.011251+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:23.011251+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37453824, 'num_objects': 9144, 'num_object_clones': 0, 'num_object_copies': 18288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9144, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9603, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3052}, {'shards': '2', 'objects': 4693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:28.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11793", 'reported_seq': 11930, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.307730+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:23.307730+0000', 'last_peered': '2024-09-05T13:17:23.307730+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:23.307730+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37343232, 'num_objects': 9117, 'num_object_clones': 0, 'num_object_copies': 18234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8160, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4663}, {'shards': '0,2', 'objects': 3120}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11619", 'reported_seq': 11742, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.097426+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:23.097426+0000', 'last_peered': '2024-09-05T13:17:23.097426+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:23.097426+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36712448, 'num_objects': 8963, 'num_object_clones': 0, 'num_object_copies': 17926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9128, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3062}, {'shards': '2', 'objects': 4576}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11830", 'reported_seq': 11970, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.305976+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:23.305976+0000', 'last_peered': '2024-09-05T13:17:23.305976+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:23.305976+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37060608, 'num_objects': 9048, 'num_object_clones': 0, 'num_object_copies': 18096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7981, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4629}, {'shards': '0,2', 'objects': 3029}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11939", 'reported_seq': 12126, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.350606+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:23.350606+0000', 'last_peered': '2024-09-05T13:17:23.350606+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:23.350606+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37539840, 'num_objects': 9165, 'num_object_clones': 0, 'num_object_copies': 18330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7838, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4721}, {'shards': '0,2', 'objects': 3061}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11926", 'reported_seq': 12064, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.385115+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:23.385115+0000', 'last_peered': '2024-09-05T13:17:23.385115+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:23.385115+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37494784, 'num_objects': 9154, 'num_object_clones': 0, 'num_object_copies': 18308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8056, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4588}, {'shards': '0,2', 'objects': 3182}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:28.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11960", 'reported_seq': 12100, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:23.229478+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:23.229478+0000', 'last_peered': '2024-09-05T13:17:23.229478+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:23.229478+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37814755, 'num_objects': 9233, 'num_object_clones': 0, 'num_object_copies': 18466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8628, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4668}, {'shards': '0,2', 'objects': 3207}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:28.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:28.839 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:28.839 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44709/291952 objects degraded (15.314%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":375,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145976,"data_bytes":597901838,"bytes_used":2243559424,"bytes_avail":287666733056,"bytes_total":289910292480,"degraded_objects":44709,"degraded_total":291952,"degraded_ratio":0.1531381870992492,"misplaced_objects":3119,"misplaced_total":291952,"misplaced_ratio":0.010683263002137337,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":58683,"recovering_keys_per_sec":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":368},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:28.864 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:28.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:29.398 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:29.426 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-05T13:17:27.624159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597901838,"num_objects":145976,"num_object_clones":0,"num_object_copies":291952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44709,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":145976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145215,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5669,"num_bytes_recovered":23207955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2190976,"kb_used_data":1247688,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":280924544,"statfs":{"total":289910292480,"available":287666733056,"internally_reserved":0,"allocated":1277632512,"data_stored":1276777499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":965856262},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,2,8,8,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18104320,"num_objects":-4420,"num_object_clones":0,"num_object_copies":-8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":-212,"num_objects_unfound":0,"num_objects_dirty":-4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005271"},"pg_stats":[{"pgid":"2.f","version":"45'11807","reported_seq":11880,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.370665+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:23.370665+0000","last_peered":"2024-09-05T13:17:23.370665+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:23.370665+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11807,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4744},{"shards":"0,2","objects":3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11687","reported_seq":11792,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.115660+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:23.115660+0000","last_peered":"2024-09-05T13:17:23.115660+0000","last_clean":"2024-09-05T13:17:23.115660+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:23.115660+0000","last_undegraded":"2024-09-05T13:17:23.115660+0000","last_fullsized":"2024-09-05T13:17:23.115660+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37138432,"num_objects":9067,"num_object_clones":0,"num_object_copies":18134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9697,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11997","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.145255+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:23.145255+0000","last_peered":"2024-09-05T13:17:23.145255+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:23.145255+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37675032,"num_objects":9199,"num_object_clones":0,"num_object_copies":18398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9808,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3095},{"shards":"2","objects":4707}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11814","reported_seq":11883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.144962+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:23.144962+0000","last_peered":"2024-09-05T13:17:23.144962+0000","last_clean":"2024-09-05T13:17:23.144962+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:23.144962+0000","last_undegraded":"2024-09-05T13:17:23.144962+0000","last_fullsized":"2024-09-05T13:17:23.144962+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37732352,"num_objects":9212,"num_object_clones":0,"num_object_copies":18424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11838","reported_seq":11971,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.348625+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:23.348625+0000","last_peered":"2024-09-05T13:17:23.348625+0000","last_clean":"2024-09-05T13:17:23.348625+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:23.348625+0000","last_undegraded":"2024-09-05T13:17:23.348625+0000","last_fullsized":"2024-09-05T13:17:23.348625+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11932","reported_seq":12070,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.279113+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:23.279113+0000","last_peered":"2024-09-05T13:17:23.279113+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:23.279113+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37777408,"num_objects":9223,"num_object_clones":0,"num_object_copies":18446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8387,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4772},{"shards":"0,2","objects":3099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11632","reported_seq":14325,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.342122+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:23.342122+0000","last_peered":"2024-09-05T13:17:23.342122+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:23.342122+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36847616,"num_objects":8996,"num_object_clones":0,"num_object_copies":17992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":8996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7951,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1276,"num_bytes_recovered":5226496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11924","reported_seq":12056,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:23.112292+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:23.112292+0000","last_peered":"2024-09-05T13:17:23.112292+0000","last_clean":"2024-09-05T13:17:23.112292+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:23.112292+0000","last_undegraded":"2024-09-05T13:17:23.112292+0000","last_fullsized":"2024-09-05T13:17:23.112292+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37601280,"num_objects":9180,"num_object_clones":0,"num_object_copies":18360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8534,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11949","reported_seq":12084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.011251+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:23.011251+0000","last_peered":"2024-09-05T13:17:23.011251+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:23.011251+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9603,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3052},{"shards":"2","objects":4693}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11793","reported_seq":11930,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.307730+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:23.307730+0000","last_peered":"2024-09-05T13:17:23.307730+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:23.307730+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8160,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":3120}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11848","reported_seq":20749,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:22.978947+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:22.978947+0000","last_peered":"2024-09-05T13:17:22.978947+0000","last_clean":"2024-09-05T13:17:22.978947+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:22.978947+0000","last_undegraded":"2024-09-05T13:17:22.978947+0000","last_fullsized":"2024-09-05T13:17:22.978947+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122048,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9443,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11619","reported_seq":11742,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.097426+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:23.097426+0000","last_peered":"2024-09-05T13:17:23.097426+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:23.097426+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3062},{"shards":"2","objects":4576}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11830","reported_seq":11970,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.305976+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:23.305976+0000","last_peered":"2024-09-05T13:17:23.305976+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:23.305976+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37060608,"num_objects":9048,"num_object_clones":0,"num_object_copies":18096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7981,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":3029}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11939","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.350606+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:23.350606+0000","last_peered":"2024-09-05T13:17:23.350606+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:23.350606+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37539840,"num_objects":9165,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7838,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4721},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11926","reported_seq":12064,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.385115+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:23.385115+0000","last_peered":"2024-09-05T13:17:23.385115+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:23.385115+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8056,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11960","reported_seq":12100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:23.229478+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:23.229478+0000","last_peered":"2024-09-05T13:17:23.229478+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:23.229478+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37814755,"num_objects":9233,"num_object_clones":0,"num_object_copies":18466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4668},{"shards":"0,2","objects":3207}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":597901819,"num_objects":145974,"num_object_clones":0,"num_object_copies":291948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44709,"num_objects_misplaced":3119,"num_objects_unfound":0,"num_objects_dirty":145974,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":145213,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5668,"num_bytes_recovered":23207936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1274671104,"data_stored":1274579861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719856,"kb_used_data":435488,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93651984,"statfs":{"total":96636764160,"available":95899631616,"internally_reserved":0,"allocated":445939712,"data_stored":445680155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885320,"kb_used_data":510768,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93486520,"statfs":{"total":96636764160,"available":95730196480,"internally_reserved":0,"allocated":523026432,"data_stored":522766870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585800,"kb_used_data":301432,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93786040,"statfs":{"total":96636764160,"available":96036904960,"internally_reserved":0,"allocated":308666368,"data_stored":308330474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":521412608,"data_stored":521408995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308285440,"data_stored":308205495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444973056,"data_stored":444965371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:32.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:33.042 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:33.043 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:33.069 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-05T13:17:31.625561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":588812814,"num_objects":143757,"num_object_clones":0,"num_object_copies":287514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44745,"num_objects_misplaced":2987,"num_objects_unfound":0,"num_objects_dirty":143757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147434,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5765,"num_bytes_recovered":23601171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161017,"ondisk_log_size":161017,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2196040,"kb_used_data":1236112,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280919480,"statfs":{"total":289910292480,"available":287661547520,"internally_reserved":0,"allocated":1265778688,"data_stored":1264923675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":982895622},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,1,11,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18231296,"num_objects":-4451,"num_object_clones":0,"num_object_copies":-8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":-259,"num_objects_unfound":0,"num_objects_dirty":-4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4451,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005349"},"pg_stats":[{"pgid":"2.f","version":"45'11951","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.382910+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:28.382910+0000","last_peered":"2024-09-05T13:17:28.382910+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:28.382910+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36950016,"num_objects":9021,"num_object_clones":0,"num_object_copies":18042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11951,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4600},{"shards":"0,2","objects":2955}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11837","reported_seq":11942,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.114408+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:28.114408+0000","last_peered":"2024-09-05T13:17:28.114408+0000","last_clean":"2024-09-05T13:17:28.114408+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:28.114408+0000","last_undegraded":"2024-09-05T13:17:28.114408+0000","last_fullsized":"2024-09-05T13:17:28.114408+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9847,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12132","reported_seq":12261,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.101864+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:28.101864+0000","last_peered":"2024-09-05T13:17:28.101864+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:28.101864+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122072,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9943,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2960},{"shards":"2","objects":4572}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11961","reported_seq":12031,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.104065+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:28.104065+0000","last_peered":"2024-09-05T13:17:28.104065+0000","last_clean":"2024-09-05T13:17:28.104065+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:28.104065+0000","last_undegraded":"2024-09-05T13:17:28.104065+0000","last_fullsized":"2024-09-05T13:17:28.104065+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37130240,"num_objects":9065,"num_object_clones":0,"num_object_copies":18130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11961,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11960","reported_seq":12095,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.468886+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:28.468886+0000","last_peered":"2024-09-05T13:17:28.468886+0000","last_clean":"2024-09-05T13:17:28.468886+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:28.468886+0000","last_undegraded":"2024-09-05T13:17:28.468886+0000","last_fullsized":"2024-09-05T13:17:28.468886+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36548608,"num_objects":8923,"num_object_clones":0,"num_object_copies":17846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8500,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12072","reported_seq":12211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.424896+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:28.424896+0000","last_peered":"2024-09-05T13:17:28.424896+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:28.424896+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37203968,"num_objects":9083,"num_object_clones":0,"num_object_copies":18166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8527,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4632},{"shards":"0,2","objects":2959}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11764","reported_seq":14649,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.424723+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:28.424723+0000","last_peered":"2024-09-05T13:17:28.424723+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:28.424723+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36306944,"num_objects":8864,"num_object_clones":0,"num_object_copies":17728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":77,"num_objects_misplaced":2987,"num_objects_unfound":0,"num_objects_dirty":8864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8083,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1372,"num_bytes_recovered":5619712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5800},{"shards":"0,2","objects":2987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12073","reported_seq":12203,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.440101+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:28.440101+0000","last_peered":"2024-09-05T13:17:28.440101+0000","last_clean":"2024-09-05T13:17:28.440101+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:28.440101+0000","last_undegraded":"2024-09-05T13:17:28.440101+0000","last_fullsized":"2024-09-05T13:17:28.440101+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36990976,"num_objects":9031,"num_object_clones":0,"num_object_copies":18062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8683,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12067","reported_seq":12202,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.118055+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:28.118055+0000","last_peered":"2024-09-05T13:17:28.118055+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:28.118055+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36970496,"num_objects":9026,"num_object_clones":0,"num_object_copies":18052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9721,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2934},{"shards":"2","objects":4575}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11936","reported_seq":12074,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.446422+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:28.446422+0000","last_peered":"2024-09-05T13:17:28.446422+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:28.446422+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36757504,"num_objects":8974,"num_object_clones":0,"num_object_copies":17948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8303,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4520},{"shards":"0,2","objects":2977}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11993","reported_seq":20894,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.139466+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:28.139466+0000","last_peered":"2024-09-05T13:17:28.139466+0000","last_clean":"2024-09-05T13:17:28.139466+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:28.139466+0000","last_undegraded":"2024-09-05T13:17:28.139466+0000","last_fullsized":"2024-09-05T13:17:28.139466+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36528128,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9588,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11760","reported_seq":11882,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.118027+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:28.118027+0000","last_peered":"2024-09-05T13:17:28.118027+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:28.118027+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9269,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2921},{"shards":"2","objects":4435}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11967","reported_seq":12108,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.470040+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:28.470040+0000","last_peered":"2024-09-05T13:17:28.470040+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:28.470040+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8118,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4492},{"shards":"0,2","objects":2892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12072","reported_seq":12260,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.472135+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:28.472135+0000","last_peered":"2024-09-05T13:17:28.472135+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:28.472135+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36995072,"num_objects":9032,"num_object_clones":0,"num_object_copies":18064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7971,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":2928}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12078","reported_seq":12217,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.409138+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:28.409138+0000","last_peered":"2024-09-05T13:17:28.409138+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:28.409138+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8208,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4436},{"shards":"0,2","objects":3030}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12091","reported_seq":12231,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.476098+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:28.476098+0000","last_peered":"2024-09-05T13:17:28.476098+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:28.476098+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37278179,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8759,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4537},{"shards":"0,2","objects":3076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":588812795,"num_objects":143755,"num_object_clones":0,"num_object_copies":287510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44745,"num_objects_misplaced":2987,"num_objects_unfound":0,"num_objects_dirty":143755,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147432,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5764,"num_bytes_recovered":23601152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1262706688,"data_stored":1262615445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719780,"kb_used_data":431252,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93652060,"statfs":{"total":96636764160,"available":95899709440,"internally_reserved":0,"allocated":441602048,"data_stored":441342491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":886828,"kb_used_data":503956,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93485012,"statfs":{"total":96636764160,"available":95728652288,"internally_reserved":0,"allocated":516050944,"data_stored":515791382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,10,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589432,"kb_used_data":300904,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93782408,"statfs":{"total":96636764160,"available":96033185792,"internally_reserved":0,"allocated":308125696,"data_stored":307789802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":514486272,"data_stored":514482659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307740672,"data_stored":307660727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":440479744,"data_stored":440472059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:33.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:33.614 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:33.616 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:33.635 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-05T13:17:31.625561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":588812814,"num_objects":143757,"num_object_clones":0,"num_object_copies":287514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44745,"num_objects_misplaced":2987,"num_objects_unfound":0,"num_objects_dirty":143757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147434,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5765,"num_bytes_recovered":23601171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161017,"ondisk_log_size":161017,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2196040,"kb_used_data":1236112,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280919480,"statfs":{"total":289910292480,"available":287661547520,"internally_reserved":0,"allocated":1265778688,"data_stored":1264923675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":982895622},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,1,11,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18231296,"num_objects":-4451,"num_object_clones":0,"num_object_copies":-8902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":-259,"num_objects_unfound":0,"num_objects_dirty":-4451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4451,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005349"},"pg_stats":[{"pgid":"2.f","version":"45'11951","reported_seq":12026,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.382910+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:28.382910+0000","last_peered":"2024-09-05T13:17:28.382910+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:28.382910+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36950016,"num_objects":9021,"num_object_clones":0,"num_object_copies":18042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11951,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4600},{"shards":"0,2","objects":2955}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11837","reported_seq":11942,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.114408+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:28.114408+0000","last_peered":"2024-09-05T13:17:28.114408+0000","last_clean":"2024-09-05T13:17:28.114408+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:28.114408+0000","last_undegraded":"2024-09-05T13:17:28.114408+0000","last_fullsized":"2024-09-05T13:17:28.114408+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36524032,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9847,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12132","reported_seq":12261,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.101864+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:28.101864+0000","last_peered":"2024-09-05T13:17:28.101864+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:28.101864+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122072,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9943,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2960},{"shards":"2","objects":4572}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11961","reported_seq":12031,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.104065+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:28.104065+0000","last_peered":"2024-09-05T13:17:28.104065+0000","last_clean":"2024-09-05T13:17:28.104065+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:28.104065+0000","last_undegraded":"2024-09-05T13:17:28.104065+0000","last_fullsized":"2024-09-05T13:17:28.104065+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37130240,"num_objects":9065,"num_object_clones":0,"num_object_copies":18130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11961,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11960","reported_seq":12095,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.468886+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:28.468886+0000","last_peered":"2024-09-05T13:17:28.468886+0000","last_clean":"2024-09-05T13:17:28.468886+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:28.468886+0000","last_undegraded":"2024-09-05T13:17:28.468886+0000","last_fullsized":"2024-09-05T13:17:28.468886+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36548608,"num_objects":8923,"num_object_clones":0,"num_object_copies":17846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8500,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12072","reported_seq":12211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.424896+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:28.424896+0000","last_peered":"2024-09-05T13:17:28.424896+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:28.424896+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37203968,"num_objects":9083,"num_object_clones":0,"num_object_copies":18166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8527,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4632},{"shards":"0,2","objects":2959}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11764","reported_seq":14649,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.424723+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:28.424723+0000","last_peered":"2024-09-05T13:17:28.424723+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:28.424723+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36306944,"num_objects":8864,"num_object_clones":0,"num_object_copies":17728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":77,"num_objects_misplaced":2987,"num_objects_unfound":0,"num_objects_dirty":8864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8083,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1372,"num_bytes_recovered":5619712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5800},{"shards":"0,2","objects":2987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12073","reported_seq":12203,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.440101+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:28.440101+0000","last_peered":"2024-09-05T13:17:28.440101+0000","last_clean":"2024-09-05T13:17:28.440101+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:28.440101+0000","last_undegraded":"2024-09-05T13:17:28.440101+0000","last_fullsized":"2024-09-05T13:17:28.440101+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36990976,"num_objects":9031,"num_object_clones":0,"num_object_copies":18062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8683,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12067","reported_seq":12202,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.118055+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:28.118055+0000","last_peered":"2024-09-05T13:17:28.118055+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:28.118055+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36970496,"num_objects":9026,"num_object_clones":0,"num_object_copies":18052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9721,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2934},{"shards":"2","objects":4575}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11936","reported_seq":12074,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.446422+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:28.446422+0000","last_peered":"2024-09-05T13:17:28.446422+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:28.446422+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36757504,"num_objects":8974,"num_object_clones":0,"num_object_copies":17948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8303,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4520},{"shards":"0,2","objects":2977}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11993","reported_seq":20894,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:28.139466+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:28.139466+0000","last_peered":"2024-09-05T13:17:28.139466+0000","last_clean":"2024-09-05T13:17:28.139466+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:28.139466+0000","last_undegraded":"2024-09-05T13:17:28.139466+0000","last_fullsized":"2024-09-05T13:17:28.139466+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36528128,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9588,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11760","reported_seq":11882,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.118027+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:28.118027+0000","last_peered":"2024-09-05T13:17:28.118027+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:28.118027+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9269,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2921},{"shards":"2","objects":4435}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11967","reported_seq":12108,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.470040+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:28.470040+0000","last_peered":"2024-09-05T13:17:28.470040+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:28.470040+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36499456,"num_objects":8911,"num_object_clones":0,"num_object_copies":17822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8118,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4492},{"shards":"0,2","objects":2892}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12072","reported_seq":12260,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.472135+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:28.472135+0000","last_peered":"2024-09-05T13:17:28.472135+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:28.472135+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36995072,"num_objects":9032,"num_object_clones":0,"num_object_copies":18064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7971,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":2928}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12078","reported_seq":12217,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.409138+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:28.409138+0000","last_peered":"2024-09-05T13:17:28.409138+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:28.409138+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36872192,"num_objects":9002,"num_object_clones":0,"num_object_copies":18004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8208,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4436},{"shards":"0,2","objects":3030}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12091","reported_seq":12231,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:28.476098+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:28.476098+0000","last_peered":"2024-09-05T13:17:28.476098+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:28.476098+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37278179,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8759,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4537},{"shards":"0,2","objects":3076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":588812795,"num_objects":143755,"num_object_clones":0,"num_object_copies":287510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44745,"num_objects_misplaced":2987,"num_objects_unfound":0,"num_objects_dirty":143755,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147432,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5764,"num_bytes_recovered":23601152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1262706688,"data_stored":1262615445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719780,"kb_used_data":431252,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93652060,"statfs":{"total":96636764160,"available":95899709440,"internally_reserved":0,"allocated":441602048,"data_stored":441342491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":886828,"kb_used_data":503956,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93485012,"statfs":{"total":96636764160,"available":95728652288,"internally_reserved":0,"allocated":516050944,"data_stored":515791382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,10,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589432,"kb_used_data":300904,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93782408,"statfs":{"total":96636764160,"available":96033185792,"internally_reserved":0,"allocated":308125696,"data_stored":307789802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":514486272,"data_stored":514482659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307740672,"data_stored":307660727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":440479744,"data_stored":440472059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:33.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:33.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11951", 'reported_seq': 12026, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.382910+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:28.382910+0000', 'last_peered': '2024-09-05T13:17:28.382910+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:28.382910+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36950016, 'num_objects': 9021, 'num_object_clones': 0, 'num_object_copies': 18042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11951, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4600}, {'shards': '0,2', 'objects': 2955}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:33.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:33.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12132", 'reported_seq': 12261, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.101864+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:28.101864+0000', 'last_peered': '2024-09-05T13:17:28.101864+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:28.101864+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37122072, 'num_objects': 9064, 'num_object_clones': 0, 'num_object_copies': 18128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9064, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9943, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2960}, {'shards': '2', 'objects': 4572}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:33.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:33.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12072", 'reported_seq': 12211, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.424896+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:28.424896+0000', 'last_peered': '2024-09-05T13:17:28.424896+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:28.424896+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37203968, 'num_objects': 9083, 'num_object_clones': 0, 'num_object_copies': 18166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9083, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8527, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4632}, {'shards': '0,2', 'objects': 2959}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:33.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:33.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11764", 'reported_seq': 14649, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.424723+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:28.424723+0000', 'last_peered': '2024-09-05T13:17:28.424723+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:28.424723+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36306944, 'num_objects': 8864, 'num_object_clones': 0, 'num_object_copies': 17728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 77, 'num_objects_misplaced': 2987, 'num_objects_unfound': 0, 'num_objects_dirty': 8864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8083, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1372, 'num_bytes_recovered': 5619712, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5800}, {'shards': '0,2', 'objects': 2987}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:33.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:33.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12067", 'reported_seq': 12202, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.118055+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:28.118055+0000', 'last_peered': '2024-09-05T13:17:28.118055+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:28.118055+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36970496, 'num_objects': 9026, 'num_object_clones': 0, 'num_object_copies': 18052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9721, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2934}, {'shards': '2', 'objects': 4575}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:33.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:33.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11936", 'reported_seq': 12074, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.446422+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:28.446422+0000', 'last_peered': '2024-09-05T13:17:28.446422+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:28.446422+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36757504, 'num_objects': 8974, 'num_object_clones': 0, 'num_object_copies': 17948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8974, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8303, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4520}, {'shards': '0,2', 'objects': 2977}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:33.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11760", 'reported_seq': 11882, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.118027+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:28.118027+0000', 'last_peered': '2024-09-05T13:17:28.118027+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:28.118027+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36134912, 'num_objects': 8822, 'num_object_clones': 0, 'num_object_copies': 17644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9269, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2921}, {'shards': '2', 'objects': 4435}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11967", 'reported_seq': 12108, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.470040+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:28.470040+0000', 'last_peered': '2024-09-05T13:17:28.470040+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:28.470040+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36499456, 'num_objects': 8911, 'num_object_clones': 0, 'num_object_copies': 17822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8911, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8118, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4492}, {'shards': '0,2', 'objects': 2892}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12072", 'reported_seq': 12260, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.472135+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:28.472135+0000', 'last_peered': '2024-09-05T13:17:28.472135+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:28.472135+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36995072, 'num_objects': 9032, 'num_object_clones': 0, 'num_object_copies': 18064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7971, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4588}, {'shards': '0,2', 'objects': 2928}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12078", 'reported_seq': 12217, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.409138+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:28.409138+0000', 'last_peered': '2024-09-05T13:17:28.409138+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:28.409138+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36872192, 'num_objects': 9002, 'num_object_clones': 0, 'num_object_copies': 18004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8208, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4436}, {'shards': '0,2', 'objects': 3030}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12091", 'reported_seq': 12231, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:28.476098+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:28.476098+0000', 'last_peered': '2024-09-05T13:17:28.476098+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:28.476098+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37278179, 'num_objects': 9102, 'num_object_clones': 0, 'num_object_copies': 18204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9102, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8759, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4537}, {'shards': '0,2', 'objects': 3076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:33.639 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:34.315 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:34.315 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44836/283066 objects degraded (15.839%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":381,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":141533,"data_bytes":579703310,"bytes_used":2228785152,"bytes_avail":287681507328,"bytes_total":289910292480,"degraded_objects":44836,"degraded_total":283066,"degraded_ratio":0.15839415542664961,"misplaced_objects":2816,"misplaced_total":283066,"misplaced_ratio":0.0099482099581016444,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":89732,"recovering_keys_per_sec":0,"num_objects_recovered":263,"num_bytes_recovered":1077248,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":556},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:34.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:34.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:34.905 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:34.905 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:34.932 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-05T13:17:33.626180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":579703310,"num_objects":141533,"num_object_clones":0,"num_object_copies":283066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44836,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":141533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149658,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":23928851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160641,"ondisk_log_size":160641,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176548,"kb_used_data":1224948,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280938972,"statfs":{"total":289910292480,"available":287681507328,"internally_reserved":0,"allocated":1254346752,"data_stored":1253491739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":974375942},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,8,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27340800,"num_objects":-6675,"num_object_clones":0,"num_object_copies":-13350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":167,"num_objects_misplaced":-430,"num_objects_unfound":0,"num_objects_dirty":-6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6675,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005144"},"pg_stats":[{"pgid":"2.f","version":"45'12091","reported_seq":12165,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.509076+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:33.509076+0000","last_peered":"2024-09-05T13:17:33.509076+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:33.509076+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36376576,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12091,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4460},{"shards":"0,2","objects":2815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11970","reported_seq":12075,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.039591+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:33.039591+0000","last_peered":"2024-09-05T13:17:33.039591+0000","last_clean":"2024-09-05T13:17:33.039591+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:33.039591+0000","last_undegraded":"2024-09-05T13:17:33.039591+0000","last_fullsized":"2024-09-05T13:17:33.039591+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35979264,"num_objects":8784,"num_object_clones":0,"num_object_copies":17568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12275","reported_seq":12403,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.159928+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:33.159928+0000","last_peered":"2024-09-05T13:17:33.159928+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:33.159928+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36536344,"num_objects":8921,"num_object_clones":0,"num_object_copies":17842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8921,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10086,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2817},{"shards":"2","objects":4429}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12112","reported_seq":12182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.086217+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:33.086217+0000","last_peered":"2024-09-05T13:17:33.086217+0000","last_clean":"2024-09-05T13:17:33.086217+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:33.086217+0000","last_undegraded":"2024-09-05T13:17:33.086217+0000","last_fullsized":"2024-09-05T13:17:33.086217+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36511744,"num_objects":8914,"num_object_clones":0,"num_object_copies":17828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12112,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12091","reported_seq":12226,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.465007+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:33.465007+0000","last_peered":"2024-09-05T13:17:33.465007+0000","last_clean":"2024-09-05T13:17:33.465007+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:33.465007+0000","last_undegraded":"2024-09-05T13:17:33.465007+0000","last_fullsized":"2024-09-05T13:17:33.465007+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012032,"num_objects":8792,"num_object_clones":0,"num_object_copies":17584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8631,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12202","reported_seq":12341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.457207+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:33.457207+0000","last_peered":"2024-09-05T13:17:33.457207+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:33.457207+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671488,"num_objects":8953,"num_object_clones":0,"num_object_copies":17906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8657,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4502},{"shards":"0,2","objects":2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11935","reported_seq":14980,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.511114+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:33.511114+0000","last_peered":"2024-09-05T13:17:33.511114+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:33.511114+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35606528,"num_objects":8693,"num_object_clones":0,"num_object_copies":17386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":8693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8254,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1452,"num_bytes_recovered":5947392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5709},{"shards":"0,2","objects":2816}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12209","reported_seq":12341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.480965+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:33.480965+0000","last_peered":"2024-09-05T13:17:33.480965+0000","last_clean":"2024-09-05T13:17:33.480965+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:33.480965+0000","last_undegraded":"2024-09-05T13:17:33.480965+0000","last_fullsized":"2024-09-05T13:17:33.480965+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8819,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12204","reported_seq":12339,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.108042+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:33.108042+0000","last_peered":"2024-09-05T13:17:33.108042+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:33.108042+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36409344,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9858,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2797},{"shards":"2","objects":4438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12073","reported_seq":12211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.457302+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:33.457302+0000","last_peered":"2024-09-05T13:17:33.457302+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:33.457302+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36196352,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8440,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4383},{"shards":"0,2","objects":2840}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12124","reported_seq":21025,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.030019+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:33.030019+0000","last_peered":"2024-09-05T13:17:33.030019+0000","last_clean":"2024-09-05T13:17:33.030019+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:33.030019+0000","last_undegraded":"2024-09-05T13:17:33.030019+0000","last_fullsized":"2024-09-05T13:17:33.030019+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35991552,"num_objects":8787,"num_object_clones":0,"num_object_copies":17574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9719,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11897","reported_seq":12020,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.147469+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:33.147469+0000","last_peered":"2024-09-05T13:17:33.147469+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:33.147469+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573760,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2784},{"shards":"2","objects":4298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12111","reported_seq":12252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.472321+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:33.472321+0000","last_peered":"2024-09-05T13:17:33.472321+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:33.472321+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35909632,"num_objects":8767,"num_object_clones":0,"num_object_copies":17534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8262,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4348},{"shards":"0,2","objects":2748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12207","reported_seq":12395,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.513539+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:33.513539+0000","last_peered":"2024-09-05T13:17:33.513539+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:33.513539+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8106,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":2793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12206","reported_seq":12345,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.379068+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:33.379068+0000","last_peered":"2024-09-05T13:17:33.379068+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:33.379068+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36347904,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8336,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4308},{"shards":"0,2","objects":2902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201225+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201225+0000","last_peered":"2024-09-05T13:16:09.201225+0000","last_clean":"2024-09-05T13:16:09.201225+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201225+0000","last_undegraded":"2024-09-05T13:16:09.201225+0000","last_fullsized":"2024-09-05T13:16:09.201225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12231","reported_seq":12371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.495482+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:33.495482+0000","last_peered":"2024-09-05T13:17:33.495482+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:33.495482+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704739,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8899,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":579703291,"num_objects":141531,"num_object_clones":0,"num_object_copies":283062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44836,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":141531,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149656,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5844,"num_bytes_recovered":23928832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1251348480,"data_stored":1251257237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160638,"ondisk_log_size":160638,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719600,"kb_used_data":426912,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93652240,"statfs":{"total":96636764160,"available":95899893760,"internally_reserved":0,"allocated":437157888,"data_stored":436898331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":867516,"kb_used_data":497132,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93504324,"statfs":{"total":96636764160,"available":95748427776,"internally_reserved":0,"allocated":509063168,"data_stored":508803606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,6,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589432,"kb_used_data":300904,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93782408,"statfs":{"total":96636764160,"available":96033185792,"internally_reserved":0,"allocated":308125696,"data_stored":307789802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":507645952,"data_stored":507642339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307740672,"data_stored":307660727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435961856,"data_stored":435954171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:37.934 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:38.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:38.507 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-05T13:17:37.627488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":579703310,"num_objects":141533,"num_object_clones":0,"num_object_copies":283066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44836,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":141533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149658,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":23928851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160641,"ondisk_log_size":160641,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2184396,"kb_used_data":1224476,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280931124,"statfs":{"total":289910292480,"available":287673470976,"internally_reserved":0,"allocated":1253863424,"data_stored":1253008411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":982895622},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,12,11,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18198528,"num_objects":-4443,"num_object_clones":0,"num_object_copies":-8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":127,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":-4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004119"},"pg_stats":[{"pgid":"2.f","version":"45'12091","reported_seq":12165,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.509076+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:33.509076+0000","last_peered":"2024-09-05T13:17:33.509076+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:33.509076+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36376576,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12091,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4460},{"shards":"0,2","objects":2815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11970","reported_seq":12075,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.039591+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:33.039591+0000","last_peered":"2024-09-05T13:17:33.039591+0000","last_clean":"2024-09-05T13:17:33.039591+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:33.039591+0000","last_undegraded":"2024-09-05T13:17:33.039591+0000","last_fullsized":"2024-09-05T13:17:33.039591+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35979264,"num_objects":8784,"num_object_clones":0,"num_object_copies":17568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12275","reported_seq":12403,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.159928+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:33.159928+0000","last_peered":"2024-09-05T13:17:33.159928+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:33.159928+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36536344,"num_objects":8921,"num_object_clones":0,"num_object_copies":17842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8921,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10086,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2817},{"shards":"2","objects":4429}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12112","reported_seq":12182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.086217+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:33.086217+0000","last_peered":"2024-09-05T13:17:33.086217+0000","last_clean":"2024-09-05T13:17:33.086217+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:33.086217+0000","last_undegraded":"2024-09-05T13:17:33.086217+0000","last_fullsized":"2024-09-05T13:17:33.086217+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36511744,"num_objects":8914,"num_object_clones":0,"num_object_copies":17828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12112,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12091","reported_seq":12226,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.465007+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:33.465007+0000","last_peered":"2024-09-05T13:17:33.465007+0000","last_clean":"2024-09-05T13:17:33.465007+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:33.465007+0000","last_undegraded":"2024-09-05T13:17:33.465007+0000","last_fullsized":"2024-09-05T13:17:33.465007+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012032,"num_objects":8792,"num_object_clones":0,"num_object_copies":17584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8631,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12202","reported_seq":12341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.457207+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:33.457207+0000","last_peered":"2024-09-05T13:17:33.457207+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:33.457207+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671488,"num_objects":8953,"num_object_clones":0,"num_object_copies":17906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8657,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4502},{"shards":"0,2","objects":2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11935","reported_seq":14980,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.511114+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:33.511114+0000","last_peered":"2024-09-05T13:17:33.511114+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:33.511114+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35606528,"num_objects":8693,"num_object_clones":0,"num_object_copies":17386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":8693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8254,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1452,"num_bytes_recovered":5947392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5709},{"shards":"0,2","objects":2816}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12209","reported_seq":12341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.480965+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:33.480965+0000","last_peered":"2024-09-05T13:17:33.480965+0000","last_clean":"2024-09-05T13:17:33.480965+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:33.480965+0000","last_undegraded":"2024-09-05T13:17:33.480965+0000","last_fullsized":"2024-09-05T13:17:33.480965+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8819,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12204","reported_seq":12339,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.108042+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:33.108042+0000","last_peered":"2024-09-05T13:17:33.108042+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:33.108042+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36409344,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9858,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2797},{"shards":"2","objects":4438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12073","reported_seq":12211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.457302+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:33.457302+0000","last_peered":"2024-09-05T13:17:33.457302+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:33.457302+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36196352,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8440,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4383},{"shards":"0,2","objects":2840}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12124","reported_seq":21025,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.030019+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:33.030019+0000","last_peered":"2024-09-05T13:17:33.030019+0000","last_clean":"2024-09-05T13:17:33.030019+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:33.030019+0000","last_undegraded":"2024-09-05T13:17:33.030019+0000","last_fullsized":"2024-09-05T13:17:33.030019+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35991552,"num_objects":8787,"num_object_clones":0,"num_object_copies":17574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9719,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11897","reported_seq":12020,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.147469+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:33.147469+0000","last_peered":"2024-09-05T13:17:33.147469+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:33.147469+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573760,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2784},{"shards":"2","objects":4298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12111","reported_seq":12252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.472321+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:33.472321+0000","last_peered":"2024-09-05T13:17:33.472321+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:33.472321+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35909632,"num_objects":8767,"num_object_clones":0,"num_object_copies":17534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8262,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4348},{"shards":"0,2","objects":2748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12207","reported_seq":12395,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.513539+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:33.513539+0000","last_peered":"2024-09-05T13:17:33.513539+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:33.513539+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8106,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":2793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12206","reported_seq":12345,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.379068+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:33.379068+0000","last_peered":"2024-09-05T13:17:33.379068+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:33.379068+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36347904,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8336,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4308},{"shards":"0,2","objects":2902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12231","reported_seq":12371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.495482+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:33.495482+0000","last_peered":"2024-09-05T13:17:33.495482+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:33.495482+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704739,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8899,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":579703291,"num_objects":141531,"num_object_clones":0,"num_object_copies":283062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44836,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":141531,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149656,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5844,"num_bytes_recovered":23928832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1250861056,"data_stored":1250769813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160638,"ondisk_log_size":160638,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719600,"kb_used_data":426912,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93652240,"statfs":{"total":96636764160,"available":95899893760,"internally_reserved":0,"allocated":437157888,"data_stored":436898331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":867516,"kb_used_data":497132,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93504324,"statfs":{"total":96636764160,"available":95748427776,"internally_reserved":0,"allocated":509063168,"data_stored":508803606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,6,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597280,"kb_used_data":300432,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93774560,"statfs":{"total":96636764160,"available":96025149440,"internally_reserved":0,"allocated":307642368,"data_stored":307306474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":303951535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":507645952,"data_stored":507642339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307253248,"data_stored":307173303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435961856,"data_stored":435954171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:38.532 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:39.079 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:39.079 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:39.114 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-05T13:17:37.627488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":579703310,"num_objects":141533,"num_object_clones":0,"num_object_copies":283066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44836,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":141533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149658,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5845,"num_bytes_recovered":23928851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160641,"ondisk_log_size":160641,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2184396,"kb_used_data":1224476,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280931124,"statfs":{"total":289910292480,"available":287673470976,"internally_reserved":0,"allocated":1253863424,"data_stored":1253008411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":982895622},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,12,11,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18198528,"num_objects":-4443,"num_object_clones":0,"num_object_copies":-8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":127,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":-4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004119"},"pg_stats":[{"pgid":"2.f","version":"45'12091","reported_seq":12165,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.509076+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:33.509076+0000","last_peered":"2024-09-05T13:17:33.509076+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:33.509076+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36376576,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12091,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4460},{"shards":"0,2","objects":2815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11970","reported_seq":12075,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.039591+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:33.039591+0000","last_peered":"2024-09-05T13:17:33.039591+0000","last_clean":"2024-09-05T13:17:33.039591+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:33.039591+0000","last_undegraded":"2024-09-05T13:17:33.039591+0000","last_fullsized":"2024-09-05T13:17:33.039591+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35979264,"num_objects":8784,"num_object_clones":0,"num_object_copies":17568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12275","reported_seq":12403,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.159928+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:33.159928+0000","last_peered":"2024-09-05T13:17:33.159928+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:33.159928+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36536344,"num_objects":8921,"num_object_clones":0,"num_object_copies":17842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8921,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10086,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2817},{"shards":"2","objects":4429}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12112","reported_seq":12182,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.086217+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:33.086217+0000","last_peered":"2024-09-05T13:17:33.086217+0000","last_clean":"2024-09-05T13:17:33.086217+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:33.086217+0000","last_undegraded":"2024-09-05T13:17:33.086217+0000","last_fullsized":"2024-09-05T13:17:33.086217+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36511744,"num_objects":8914,"num_object_clones":0,"num_object_copies":17828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12112,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12091","reported_seq":12226,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.465007+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:33.465007+0000","last_peered":"2024-09-05T13:17:33.465007+0000","last_clean":"2024-09-05T13:17:33.465007+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:33.465007+0000","last_undegraded":"2024-09-05T13:17:33.465007+0000","last_fullsized":"2024-09-05T13:17:33.465007+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012032,"num_objects":8792,"num_object_clones":0,"num_object_copies":17584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8631,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12202","reported_seq":12341,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.457207+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:33.457207+0000","last_peered":"2024-09-05T13:17:33.457207+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:33.457207+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671488,"num_objects":8953,"num_object_clones":0,"num_object_copies":17906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8657,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4502},{"shards":"0,2","objects":2829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11935","reported_seq":14980,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.511114+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:33.511114+0000","last_peered":"2024-09-05T13:17:33.511114+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:33.511114+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35606528,"num_objects":8693,"num_object_clones":0,"num_object_copies":17386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":8693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8254,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1452,"num_bytes_recovered":5947392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5709},{"shards":"0,2","objects":2816}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12209","reported_seq":12341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.480965+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:33.480965+0000","last_peered":"2024-09-05T13:17:33.480965+0000","last_clean":"2024-09-05T13:17:33.480965+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:33.480965+0000","last_undegraded":"2024-09-05T13:17:33.480965+0000","last_fullsized":"2024-09-05T13:17:33.480965+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8819,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12204","reported_seq":12339,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.108042+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:33.108042+0000","last_peered":"2024-09-05T13:17:33.108042+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:33.108042+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36409344,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9858,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2797},{"shards":"2","objects":4438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12073","reported_seq":12211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.457302+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:33.457302+0000","last_peered":"2024-09-05T13:17:33.457302+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:33.457302+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36196352,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8440,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4383},{"shards":"0,2","objects":2840}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12124","reported_seq":21025,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:33.030019+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:33.030019+0000","last_peered":"2024-09-05T13:17:33.030019+0000","last_clean":"2024-09-05T13:17:33.030019+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:33.030019+0000","last_undegraded":"2024-09-05T13:17:33.030019+0000","last_fullsized":"2024-09-05T13:17:33.030019+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35991552,"num_objects":8787,"num_object_clones":0,"num_object_copies":17574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9719,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11897","reported_seq":12020,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.147469+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:33.147469+0000","last_peered":"2024-09-05T13:17:33.147469+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:33.147469+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573760,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9406,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2784},{"shards":"2","objects":4298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12111","reported_seq":12252,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.472321+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:33.472321+0000","last_peered":"2024-09-05T13:17:33.472321+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:33.472321+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35909632,"num_objects":8767,"num_object_clones":0,"num_object_copies":17534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8262,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4348},{"shards":"0,2","objects":2748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12207","reported_seq":12395,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.513539+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:33.513539+0000","last_peered":"2024-09-05T13:17:33.513539+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:33.513539+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8106,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":2793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12206","reported_seq":12345,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.379068+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:33.379068+0000","last_peered":"2024-09-05T13:17:33.379068+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:33.379068+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36347904,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8336,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4308},{"shards":"0,2","objects":2902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12231","reported_seq":12371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:33.495482+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:33.495482+0000","last_peered":"2024-09-05T13:17:33.495482+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:33.495482+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704739,"num_objects":8962,"num_object_clones":0,"num_object_copies":17924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8899,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":579703291,"num_objects":141531,"num_object_clones":0,"num_object_copies":283062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44836,"num_objects_misplaced":2816,"num_objects_unfound":0,"num_objects_dirty":141531,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149656,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5844,"num_bytes_recovered":23928832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1250861056,"data_stored":1250769813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160638,"ondisk_log_size":160638,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":719600,"kb_used_data":426912,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93652240,"statfs":{"total":96636764160,"available":95899893760,"internally_reserved":0,"allocated":437157888,"data_stored":436898331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":867516,"kb_used_data":497132,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93504324,"statfs":{"total":96636764160,"available":95748427776,"internally_reserved":0,"allocated":509063168,"data_stored":508803606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,6,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597280,"kb_used_data":300432,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93774560,"statfs":{"total":96636764160,"available":96025149440,"internally_reserved":0,"allocated":307642368,"data_stored":307306474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":303951535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":507645952,"data_stored":507642339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307253248,"data_stored":307173303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435961856,"data_stored":435954171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:39.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:39.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12091", 'reported_seq': 12165, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.509076+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:33.509076+0000', 'last_peered': '2024-09-05T13:17:33.509076+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:33.509076+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36376576, 'num_objects': 8881, 'num_object_clones': 0, 'num_object_copies': 17762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12091, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4460}, {'shards': '0,2', 'objects': 2815}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:39.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:39.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12275", 'reported_seq': 12403, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.159928+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:33.159928+0000', 'last_peered': '2024-09-05T13:17:33.159928+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:33.159928+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36536344, 'num_objects': 8921, 'num_object_clones': 0, 'num_object_copies': 17842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8921, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10086, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2817}, {'shards': '2', 'objects': 4429}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12202", 'reported_seq': 12341, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.457207+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:33.457207+0000', 'last_peered': '2024-09-05T13:17:33.457207+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:33.457207+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36671488, 'num_objects': 8953, 'num_object_clones': 0, 'num_object_copies': 17906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8657, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4502}, {'shards': '0,2', 'objects': 2829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11935", 'reported_seq': 14980, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.511114+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:33.511114+0000', 'last_peered': '2024-09-05T13:17:33.511114+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:33.511114+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35606528, 'num_objects': 8693, 'num_object_clones': 0, 'num_object_copies': 17386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 168, 'num_objects_misplaced': 2816, 'num_objects_unfound': 0, 'num_objects_dirty': 8693, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8254, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1452, 'num_bytes_recovered': 5947392, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5709}, {'shards': '0,2', 'objects': 2816}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12204", 'reported_seq': 12339, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.108042+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:33.108042+0000', 'last_peered': '2024-09-05T13:17:33.108042+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:33.108042+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36409344, 'num_objects': 8889, 'num_object_clones': 0, 'num_object_copies': 17778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9858, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2797}, {'shards': '2', 'objects': 4438}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12073", 'reported_seq': 12211, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.457302+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:33.457302+0000', 'last_peered': '2024-09-05T13:17:33.457302+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:33.457302+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36196352, 'num_objects': 8837, 'num_object_clones': 0, 'num_object_copies': 17674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8440, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4383}, {'shards': '0,2', 'objects': 2840}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'11897", 'reported_seq': 12020, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.147469+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:33.147469+0000', 'last_peered': '2024-09-05T13:17:33.147469+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:33.147469+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35573760, 'num_objects': 8685, 'num_object_clones': 0, 'num_object_copies': 17370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8685, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9406, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2784}, {'shards': '2', 'objects': 4298}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:39.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:39.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12111", 'reported_seq': 12252, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.472321+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:33.472321+0000', 'last_peered': '2024-09-05T13:17:33.472321+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:33.472321+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35909632, 'num_objects': 8767, 'num_object_clones': 0, 'num_object_copies': 17534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8262, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4348}, {'shards': '0,2', 'objects': 2748}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:39.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:39.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12207", 'reported_seq': 12395, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.513539+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:33.513539+0000', 'last_peered': '2024-09-05T13:17:33.513539+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:33.513539+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36442112, 'num_objects': 8897, 'num_object_clones': 0, 'num_object_copies': 17794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8106, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4453}, {'shards': '0,2', 'objects': 2793}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:39.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:39.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12206", 'reported_seq': 12345, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.379068+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:33.379068+0000', 'last_peered': '2024-09-05T13:17:33.379068+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:33.379068+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36347904, 'num_objects': 8874, 'num_object_clones': 0, 'num_object_copies': 17748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8874, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8336, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4308}, {'shards': '0,2', 'objects': 2902}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:39.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:39.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12231", 'reported_seq': 12371, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:33.495482+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:33.495482+0000', 'last_peered': '2024-09-05T13:17:33.495482+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:33.495482+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36704739, 'num_objects': 8962, 'num_object_clones': 0, 'num_object_copies': 17924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8899, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4397}, {'shards': '0,2', 'objects': 2936}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:39.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:39.832 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:39.832 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44836/283066 objects degraded (15.839%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":386,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":141533,"data_bytes":579703310,"bytes_used":2236821504,"bytes_avail":287673470976,"bytes_total":289910292480,"degraded_objects":44836,"degraded_total":283066,"degraded_ratio":0.15839415542664961,"misplaced_objects":2816,"misplaced_total":283066,"misplaced_ratio":0.0099482099581016444,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":60054,"recovering_keys_per_sec":0,"num_objects_recovered":176,"num_bytes_recovered":720896,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":370},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:39.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:39.863 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:40.419 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:40.441 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-05T13:17:39.628187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":570728974,"num_objects":139342,"num_object_clones":0,"num_object_copies":278684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44878,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":139342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151849,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5934,"num_bytes_recovered":24293395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2164424,"kb_used_data":1212820,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280951096,"statfs":{"total":289910292480,"available":287693922304,"internally_reserved":0,"allocated":1241927680,"data_stored":1241072667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":974375942},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,9,10,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27172864,"num_objects":-6634,"num_object_clones":0,"num_object_copies":-13268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":169,"num_objects_misplaced":-434,"num_objects_unfound":0,"num_objects_dirty":-6634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":1085440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004028"},"pg_stats":[{"pgid":"2.f","version":"45'12228","reported_seq":12303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.457456+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:38.457456+0000","last_peered":"2024-09-05T13:17:38.457456+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:38.457456+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12228,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4323},{"shards":"0,2","objects":2678}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12099","reported_seq":12203,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.154513+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:38.154513+0000","last_peered":"2024-09-05T13:17:38.154513+0000","last_clean":"2024-09-05T13:17:38.154513+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:38.154513+0000","last_undegraded":"2024-09-05T13:17:38.154513+0000","last_fullsized":"2024-09-05T13:17:38.154513+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35450880,"num_objects":8655,"num_object_clones":0,"num_object_copies":17310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10109,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12423","reported_seq":12552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.192271+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:38.192271+0000","last_peered":"2024-09-05T13:17:38.192271+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:38.192271+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35930136,"num_objects":8773,"num_object_clones":0,"num_object_copies":17546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10234,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2669},{"shards":"2","objects":4281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12253","reported_seq":12323,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.172299+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:38.172299+0000","last_peered":"2024-09-05T13:17:38.172299+0000","last_clean":"2024-09-05T13:17:38.172299+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:38.172299+0000","last_undegraded":"2024-09-05T13:17:38.172299+0000","last_fullsized":"2024-09-05T13:17:38.172299+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35934208,"num_objects":8773,"num_object_clones":0,"num_object_copies":17546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12253,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12217","reported_seq":12352,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.535207+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:38.535207+0000","last_peered":"2024-09-05T13:17:38.535207+0000","last_clean":"2024-09-05T13:17:38.535207+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:38.535207+0000","last_undegraded":"2024-09-05T13:17:38.535207+0000","last_fullsized":"2024-09-05T13:17:38.535207+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35495936,"num_objects":8666,"num_object_clones":0,"num_object_copies":17332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8757,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12349","reported_seq":12488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.446009+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:38.446009+0000","last_peered":"2024-09-05T13:17:38.446009+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:38.446009+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36069376,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8804,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4355},{"shards":"0,2","objects":2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12066","reported_seq":15289,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.572357+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:38.572357+0000","last_peered":"2024-09-05T13:17:38.572357+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:38.572357+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35069952,"num_objects":8562,"num_object_clones":0,"num_object_copies":17124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":210,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":8562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":6311936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":2685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12353","reported_seq":12485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.520749+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:38.520749+0000","last_peered":"2024-09-05T13:17:38.520749+0000","last_clean":"2024-09-05T13:17:38.520749+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:38.520749+0000","last_undegraded":"2024-09-05T13:17:38.520749+0000","last_fullsized":"2024-09-05T13:17:38.520749+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35844096,"num_objects":8751,"num_object_clones":0,"num_object_copies":17502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12353","reported_seq":12488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.137585+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:38.137585+0000","last_peered":"2024-09-05T13:17:38.137585+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:38.137585+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35799040,"num_objects":8740,"num_object_clones":0,"num_object_copies":17480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2648},{"shards":"2","objects":4289}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12195","reported_seq":12333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.538944+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:38.538944+0000","last_peered":"2024-09-05T13:17:38.538944+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:38.538944+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35696640,"num_objects":8715,"num_object_clones":0,"num_object_copies":17430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8562,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4261},{"shards":"0,2","objects":2718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12247","reported_seq":21148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.122171+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:38.122171+0000","last_peered":"2024-09-05T13:17:38.122171+0000","last_clean":"2024-09-05T13:17:38.122171+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:38.122171+0000","last_undegraded":"2024-09-05T13:17:38.122171+0000","last_fullsized":"2024-09-05T13:17:38.122171+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35487744,"num_objects":8664,"num_object_clones":0,"num_object_copies":17328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9842,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12048","reported_seq":12171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.096527+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:38.096527+0000","last_peered":"2024-09-05T13:17:38.096527+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:38.096527+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2633},{"shards":"2","objects":4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12238","reported_seq":12376,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.558186+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:38.558186+0000","last_peered":"2024-09-05T13:17:38.558186+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:38.558186+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35389440,"num_objects":8640,"num_object_clones":0,"num_object_copies":17280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8389,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4221},{"shards":"0,2","objects":2621}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12348","reported_seq":12536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.510201+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:38.510201+0000","last_peered":"2024-09-05T13:17:38.510201+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:38.510201+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8247,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4312},{"shards":"0,2","objects":2652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12339","reported_seq":12478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.552979+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:38.552979+0000","last_peered":"2024-09-05T13:17:38.552979+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:38.552979+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35803136,"num_objects":8741,"num_object_clones":0,"num_object_copies":17482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8469,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4175},{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12373","reported_seq":12513,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.570013+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:38.570013+0000","last_peered":"2024-09-05T13:17:38.570013+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:38.570013+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36123107,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":2794}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":570728955,"num_objects":139340,"num_object_clones":0,"num_object_copies":278680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44878,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":139340,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151847,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5933,"num_bytes_recovered":24293376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1239633920,"data_stored":1239542677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701904,"kb_used_data":421692,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93669936,"statfs":{"total":96636764160,"available":95918014464,"internally_reserved":0,"allocated":431812608,"data_stored":431553051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865240,"kb_used_data":490696,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93506600,"statfs":{"total":96636764160,"available":95750758400,"internally_reserved":0,"allocated":502472704,"data_stored":502213142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597280,"kb_used_data":300432,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93774560,"statfs":{"total":96636764160,"available":96025149440,"internally_reserved":0,"allocated":307642368,"data_stored":307306474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":303951535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":500957184,"data_stored":500953571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307253248,"data_stored":307173303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431423488,"data_stored":431415803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:43.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:44.023 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:44.026 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:44.048 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-05T13:17:43.629412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567427598,"num_objects":138536,"num_object_clones":0,"num_object_copies":277072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44878,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":138536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152655,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5934,"num_bytes_recovered":24293395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2151200,"kb_used_data":1207912,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":280964320,"statfs":{"total":289910292480,"available":287707463680,"internally_reserved":0,"allocated":1236901888,"data_stored":1236046875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":965856262},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,10,13,15,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21385216,"num_objects":-5221,"num_object_clones":0,"num_object_copies":-10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":133,"num_objects_misplaced":-302,"num_objects_unfound":0,"num_objects_dirty":-5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5221,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003850"},"pg_stats":[{"pgid":"2.f","version":"45'12228","reported_seq":12303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.457456+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:38.457456+0000","last_peered":"2024-09-05T13:17:38.457456+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:38.457456+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12228,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4323},{"shards":"0,2","objects":2678}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12250","reported_seq":12355,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.143077+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:43.143077+0000","last_peered":"2024-09-05T13:17:43.143077+0000","last_clean":"2024-09-05T13:17:43.143077+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:43.143077+0000","last_undegraded":"2024-09-05T13:17:43.143077+0000","last_fullsized":"2024-09-05T13:17:43.143077+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12569","reported_seq":12698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.204291+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:43.204291+0000","last_peered":"2024-09-05T13:17:43.204291+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:43.204291+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332120,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10380,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2523},{"shards":"2","objects":4135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12393","reported_seq":12463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.120958+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:43.120958+0000","last_peered":"2024-09-05T13:17:43.120958+0000","last_clean":"2024-09-05T13:17:43.120958+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:43.120958+0000","last_undegraded":"2024-09-05T13:17:43.120958+0000","last_fullsized":"2024-09-05T13:17:43.120958+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12393,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12217","reported_seq":12352,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.535207+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:38.535207+0000","last_peered":"2024-09-05T13:17:38.535207+0000","last_clean":"2024-09-05T13:17:38.535207+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:38.535207+0000","last_undegraded":"2024-09-05T13:17:38.535207+0000","last_fullsized":"2024-09-05T13:17:38.535207+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35495936,"num_objects":8666,"num_object_clones":0,"num_object_copies":17332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8757,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12349","reported_seq":12488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.446009+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:38.446009+0000","last_peered":"2024-09-05T13:17:38.446009+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:38.446009+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36069376,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8804,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4355},{"shards":"0,2","objects":2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12066","reported_seq":15289,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.572357+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:38.572357+0000","last_peered":"2024-09-05T13:17:38.572357+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:38.572357+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35069952,"num_objects":8562,"num_object_clones":0,"num_object_copies":17124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":210,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":8562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":6311936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":2685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12353","reported_seq":12485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.520749+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:38.520749+0000","last_peered":"2024-09-05T13:17:38.520749+0000","last_clean":"2024-09-05T13:17:38.520749+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:38.520749+0000","last_undegraded":"2024-09-05T13:17:38.520749+0000","last_fullsized":"2024-09-05T13:17:38.520749+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35844096,"num_objects":8751,"num_object_clones":0,"num_object_copies":17502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12487","reported_seq":12622,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.080376+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:43.080376+0000","last_peered":"2024-09-05T13:17:43.080376+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:43.080376+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250176,"num_objects":8606,"num_object_clones":0,"num_object_copies":17212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10141,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2514},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12195","reported_seq":12333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.538944+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:38.538944+0000","last_peered":"2024-09-05T13:17:38.538944+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:38.538944+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35696640,"num_objects":8715,"num_object_clones":0,"num_object_copies":17430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8562,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4261},{"shards":"0,2","objects":2718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12373","reported_seq":21274,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.147886+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:43.147886+0000","last_peered":"2024-09-05T13:17:43.147886+0000","last_clean":"2024-09-05T13:17:43.147886+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:43.147886+0000","last_undegraded":"2024-09-05T13:17:43.147886+0000","last_fullsized":"2024-09-05T13:17:43.147886+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34971648,"num_objects":8538,"num_object_clones":0,"num_object_copies":17076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9968,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12157","reported_seq":12280,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.105725+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:43.105725+0000","last_peered":"2024-09-05T13:17:43.105725+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:43.105725+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34508800,"num_objects":8425,"num_object_clones":0,"num_object_copies":16850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9666,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2524},{"shards":"2","objects":4038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12238","reported_seq":12376,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.558186+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:38.558186+0000","last_peered":"2024-09-05T13:17:38.558186+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:38.558186+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35389440,"num_objects":8640,"num_object_clones":0,"num_object_copies":17280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8389,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4221},{"shards":"0,2","objects":2621}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12348","reported_seq":12536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.510201+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:38.510201+0000","last_peered":"2024-09-05T13:17:38.510201+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:38.510201+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8247,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4312},{"shards":"0,2","objects":2652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12339","reported_seq":12478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.552979+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:38.552979+0000","last_peered":"2024-09-05T13:17:38.552979+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:38.552979+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35803136,"num_objects":8741,"num_object_clones":0,"num_object_copies":17482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8469,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4175},{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12373","reported_seq":12513,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.570013+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:38.570013+0000","last_peered":"2024-09-05T13:17:38.570013+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:38.570013+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36123107,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":2794}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":567427579,"num_objects":138534,"num_object_clones":0,"num_object_copies":277068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44878,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":138534,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152653,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5933,"num_bytes_recovered":24293376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1234575360,"data_stored":1234484117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701648,"kb_used_data":417276,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93670192,"statfs":{"total":96636764160,"available":95918276608,"internally_reserved":0,"allocated":427290624,"data_stored":427031067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865240,"kb_used_data":490696,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93506600,"statfs":{"total":96636764160,"available":95750758400,"internally_reserved":0,"allocated":502472704,"data_stored":502213142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584312,"kb_used_data":299940,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93787528,"statfs":{"total":96636764160,"available":96038428672,"internally_reserved":0,"allocated":307138560,"data_stored":306802666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":500957184,"data_stored":500953571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306728960,"data_stored":306649015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426889216,"data_stored":426881531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:44.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:44.640 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:44.641 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:44.666 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-05T13:17:43.629412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567427598,"num_objects":138536,"num_object_clones":0,"num_object_copies":277072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44878,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":138536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152655,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5934,"num_bytes_recovered":24293395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2151200,"kb_used_data":1207912,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":280964320,"statfs":{"total":289910292480,"available":287707463680,"internally_reserved":0,"allocated":1236901888,"data_stored":1236046875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":965856262},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,10,13,15,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21385216,"num_objects":-5221,"num_object_clones":0,"num_object_copies":-10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":133,"num_objects_misplaced":-302,"num_objects_unfound":0,"num_objects_dirty":-5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5221,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003850"},"pg_stats":[{"pgid":"2.f","version":"45'12228","reported_seq":12303,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.457456+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:38.457456+0000","last_peered":"2024-09-05T13:17:38.457456+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:38.457456+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12228,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4323},{"shards":"0,2","objects":2678}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12250","reported_seq":12355,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.143077+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:43.143077+0000","last_peered":"2024-09-05T13:17:43.143077+0000","last_clean":"2024-09-05T13:17:43.143077+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:43.143077+0000","last_undegraded":"2024-09-05T13:17:43.143077+0000","last_fullsized":"2024-09-05T13:17:43.143077+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12569","reported_seq":12698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.204291+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:43.204291+0000","last_peered":"2024-09-05T13:17:43.204291+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:43.204291+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332120,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10380,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2523},{"shards":"2","objects":4135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12393","reported_seq":12463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.120958+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:43.120958+0000","last_peered":"2024-09-05T13:17:43.120958+0000","last_clean":"2024-09-05T13:17:43.120958+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:43.120958+0000","last_undegraded":"2024-09-05T13:17:43.120958+0000","last_fullsized":"2024-09-05T13:17:43.120958+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12393,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12217","reported_seq":12352,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.535207+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:38.535207+0000","last_peered":"2024-09-05T13:17:38.535207+0000","last_clean":"2024-09-05T13:17:38.535207+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:38.535207+0000","last_undegraded":"2024-09-05T13:17:38.535207+0000","last_fullsized":"2024-09-05T13:17:38.535207+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35495936,"num_objects":8666,"num_object_clones":0,"num_object_copies":17332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8757,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12349","reported_seq":12488,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.446009+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:38.446009+0000","last_peered":"2024-09-05T13:17:38.446009+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:38.446009+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36069376,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8804,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4355},{"shards":"0,2","objects":2682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12066","reported_seq":15289,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.572357+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:38.572357+0000","last_peered":"2024-09-05T13:17:38.572357+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:38.572357+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35069952,"num_objects":8562,"num_object_clones":0,"num_object_copies":17124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":210,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":8562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8385,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":6311936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":2685}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12353","reported_seq":12485,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:38.520749+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:38.520749+0000","last_peered":"2024-09-05T13:17:38.520749+0000","last_clean":"2024-09-05T13:17:38.520749+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:38.520749+0000","last_undegraded":"2024-09-05T13:17:38.520749+0000","last_fullsized":"2024-09-05T13:17:38.520749+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35844096,"num_objects":8751,"num_object_clones":0,"num_object_copies":17502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12487","reported_seq":12622,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.080376+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:43.080376+0000","last_peered":"2024-09-05T13:17:43.080376+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:43.080376+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250176,"num_objects":8606,"num_object_clones":0,"num_object_copies":17212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10141,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2514},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12195","reported_seq":12333,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.538944+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:38.538944+0000","last_peered":"2024-09-05T13:17:38.538944+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:38.538944+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35696640,"num_objects":8715,"num_object_clones":0,"num_object_copies":17430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8562,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4261},{"shards":"0,2","objects":2718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12373","reported_seq":21274,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.147886+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:43.147886+0000","last_peered":"2024-09-05T13:17:43.147886+0000","last_clean":"2024-09-05T13:17:43.147886+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:43.147886+0000","last_undegraded":"2024-09-05T13:17:43.147886+0000","last_fullsized":"2024-09-05T13:17:43.147886+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34971648,"num_objects":8538,"num_object_clones":0,"num_object_copies":17076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9968,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12157","reported_seq":12280,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.105725+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:43.105725+0000","last_peered":"2024-09-05T13:17:43.105725+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:43.105725+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34508800,"num_objects":8425,"num_object_clones":0,"num_object_copies":16850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9666,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2524},{"shards":"2","objects":4038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12238","reported_seq":12376,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.558186+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:38.558186+0000","last_peered":"2024-09-05T13:17:38.558186+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:38.558186+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35389440,"num_objects":8640,"num_object_clones":0,"num_object_copies":17280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8389,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4221},{"shards":"0,2","objects":2621}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12348","reported_seq":12536,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.510201+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:38.510201+0000","last_peered":"2024-09-05T13:17:38.510201+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:38.510201+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8247,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4312},{"shards":"0,2","objects":2652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12339","reported_seq":12478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.552979+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:38.552979+0000","last_peered":"2024-09-05T13:17:38.552979+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:38.552979+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35803136,"num_objects":8741,"num_object_clones":0,"num_object_copies":17482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8469,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4175},{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12373","reported_seq":12513,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:38.570013+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:38.570013+0000","last_peered":"2024-09-05T13:17:38.570013+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:38.570013+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36123107,"num_objects":8820,"num_object_clones":0,"num_object_copies":17640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":2794}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":567427579,"num_objects":138534,"num_object_clones":0,"num_object_copies":277068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44878,"num_objects_misplaced":2685,"num_objects_unfound":0,"num_objects_dirty":138534,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152653,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5933,"num_bytes_recovered":24293376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1234575360,"data_stored":1234484117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701648,"kb_used_data":417276,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93670192,"statfs":{"total":96636764160,"available":95918276608,"internally_reserved":0,"allocated":427290624,"data_stored":427031067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865240,"kb_used_data":490696,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93506600,"statfs":{"total":96636764160,"available":95750758400,"internally_reserved":0,"allocated":502472704,"data_stored":502213142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584312,"kb_used_data":299940,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93787528,"statfs":{"total":96636764160,"available":96038428672,"internally_reserved":0,"allocated":307138560,"data_stored":306802666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":500957184,"data_stored":500953571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306728960,"data_stored":306649015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426889216,"data_stored":426881531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:44.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:44.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12228", 'reported_seq': 12303, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:38.457456+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:38.457456+0000', 'last_peered': '2024-09-05T13:17:38.457456+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:38.457456+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35815424, 'num_objects': 8744, 'num_object_clones': 0, 'num_object_copies': 17488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12228, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4323}, {'shards': '0,2', 'objects': 2678}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:44.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:44.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12569", 'reported_seq': 12698, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:43.204291+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:43.204291+0000', 'last_peered': '2024-09-05T13:17:43.204291+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:43.204291+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35332120, 'num_objects': 8627, 'num_object_clones': 0, 'num_object_copies': 17254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8627, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10380, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2523}, {'shards': '2', 'objects': 4135}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:44.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:44.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12349", 'reported_seq': 12488, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:38.446009+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:38.446009+0000', 'last_peered': '2024-09-05T13:17:38.446009+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:38.446009+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36069376, 'num_objects': 8806, 'num_object_clones': 0, 'num_object_copies': 17612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8806, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8804, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4355}, {'shards': '0,2', 'objects': 2682}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:44.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:44.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12066", 'reported_seq': 15289, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:38.572357+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:38.572357+0000', 'last_peered': '2024-09-05T13:17:38.572357+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:38.572357+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35069952, 'num_objects': 8562, 'num_object_clones': 0, 'num_object_copies': 17124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 210, 'num_objects_misplaced': 2685, 'num_objects_unfound': 0, 'num_objects_dirty': 8562, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8385, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1541, 'num_bytes_recovered': 6311936, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5667}, {'shards': '0,2', 'objects': 2685}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:44.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:44.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12487", 'reported_seq': 12622, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:43.080376+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:43.080376+0000', 'last_peered': '2024-09-05T13:17:43.080376+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:43.080376+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35250176, 'num_objects': 8606, 'num_object_clones': 0, 'num_object_copies': 17212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10141, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2514}, {'shards': '2', 'objects': 4155}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:44.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:44.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12195", 'reported_seq': 12333, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:38.538944+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:38.538944+0000', 'last_peered': '2024-09-05T13:17:38.538944+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:38.538944+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35696640, 'num_objects': 8715, 'num_object_clones': 0, 'num_object_copies': 17430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8715, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8562, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4261}, {'shards': '0,2', 'objects': 2718}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:44.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:44.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'12157", 'reported_seq': 12280, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:43.105725+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:43.105725+0000', 'last_peered': '2024-09-05T13:17:43.105725+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:43.105725+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34508800, 'num_objects': 8425, 'num_object_clones': 0, 'num_object_copies': 16850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9666, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2524}, {'shards': '2', 'objects': 4038}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:44.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:44.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12238", 'reported_seq': 12376, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:38.558186+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:38.558186+0000', 'last_peered': '2024-09-05T13:17:38.558186+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:38.558186+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35389440, 'num_objects': 8640, 'num_object_clones': 0, 'num_object_copies': 17280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8640, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8389, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4221}, {'shards': '0,2', 'objects': 2621}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:44.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:44.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12348", 'reported_seq': 12536, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:38.510201+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:38.510201+0000', 'last_peered': '2024-09-05T13:17:38.510201+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:38.510201+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35864576, 'num_objects': 8756, 'num_object_clones': 0, 'num_object_copies': 17512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8247, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4312}, {'shards': '0,2', 'objects': 2652}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:44.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:44.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12339", 'reported_seq': 12478, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:38.552979+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:38.552979+0000', 'last_peered': '2024-09-05T13:17:38.552979+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:38.552979+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35803136, 'num_objects': 8741, 'num_object_clones': 0, 'num_object_copies': 17482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8469, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4175}, {'shards': '0,2', 'objects': 2769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:44.671 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:44.671 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12373", 'reported_seq': 12513, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:38.570013+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:38.570013+0000', 'last_peered': '2024-09-05T13:17:38.570013+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:38.570013+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36123107, 'num_objects': 8820, 'num_object_clones': 0, 'num_object_copies': 17640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8820, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9041, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4255}, {'shards': '0,2', 'objects': 2794}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:44.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:45.342 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:45.342 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44878/277072 objects degraded (16.197%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":392,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138536,"data_bytes":567427598,"bytes_used":2202828800,"bytes_avail":287707463680,"bytes_total":289910292480,"degraded_objects":44878,"degraded_total":277072,"degraded_ratio":0.16197233931974359,"misplaced_objects":2685,"misplaced_total":277072,"misplaced_ratio":0.0096906219322053476,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":57666,"recovering_keys_per_sec":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":434},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:45.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:45.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:45.923 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:45.924 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:45.949 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-05T13:17:45.630243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":561586702,"num_objects":137110,"num_object_clones":0,"num_object_copies":274220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44938,"num_objects_misplaced":2537,"num_objects_unfound":0,"num_objects_dirty":137110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154081,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6022,"num_bytes_recovered":24653843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160964,"ondisk_log_size":160964,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2152700,"kb_used_data":1201084,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280962820,"statfs":{"total":289910292480,"available":287705927680,"internally_reserved":0,"allocated":1229910016,"data_stored":1229055003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":974375942},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,9,16,12,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18116608,"num_objects":-4423,"num_object_clones":0,"num_object_copies":-8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":102,"num_objects_misplaced":-279,"num_objects_unfound":0,"num_objects_dirty":-4423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4423,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004062"},"pg_stats":[{"pgid":"2.f","version":"45'12367","reported_seq":12442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.528473+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:43.528473+0000","last_peered":"2024-09-05T13:17:43.528473+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:43.528473+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35246080,"num_objects":8605,"num_object_clones":0,"num_object_copies":17210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12367,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4184},{"shards":"0,2","objects":2539}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12250","reported_seq":12355,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.143077+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:43.143077+0000","last_peered":"2024-09-05T13:17:43.143077+0000","last_clean":"2024-09-05T13:17:43.143077+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:43.143077+0000","last_undegraded":"2024-09-05T13:17:43.143077+0000","last_fullsized":"2024-09-05T13:17:43.143077+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12569","reported_seq":12698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.204291+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:43.204291+0000","last_peered":"2024-09-05T13:17:43.204291+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:43.204291+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332120,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10380,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2523},{"shards":"2","objects":4135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12393","reported_seq":12463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.120958+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:43.120958+0000","last_peered":"2024-09-05T13:17:43.120958+0000","last_clean":"2024-09-05T13:17:43.120958+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:43.120958+0000","last_undegraded":"2024-09-05T13:17:43.120958+0000","last_fullsized":"2024-09-05T13:17:43.120958+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12393,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12355","reported_seq":12490,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.552929+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:43.552929+0000","last_peered":"2024-09-05T13:17:43.552929+0000","last_clean":"2024-09-05T13:17:43.552929+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:43.552929+0000","last_undegraded":"2024-09-05T13:17:43.552929+0000","last_fullsized":"2024-09-05T13:17:43.552929+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8895,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12494","reported_seq":12633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.602425+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:43.602425+0000","last_peered":"2024-09-05T13:17:43.602425+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:43.602425+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35475456,"num_objects":8661,"num_object_clones":0,"num_object_copies":17322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8949,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4210},{"shards":"0,2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12214","reported_seq":15613,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.587785+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:43.587785+0000","last_peered":"2024-09-05T13:17:43.587785+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:43.587785+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":270,"num_objects_misplaced":2537,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8533,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1629,"num_bytes_recovered":6672384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5607},{"shards":"0,2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12516","reported_seq":12648,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.534188+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:43.534188+0000","last_peered":"2024-09-05T13:17:43.534188+0000","last_clean":"2024-09-05T13:17:43.534188+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:43.534188+0000","last_undegraded":"2024-09-05T13:17:43.534188+0000","last_fullsized":"2024-09-05T13:17:43.534188+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35176448,"num_objects":8588,"num_object_clones":0,"num_object_copies":17176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9126,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12487","reported_seq":12622,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.080376+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:43.080376+0000","last_peered":"2024-09-05T13:17:43.080376+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:43.080376+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250176,"num_objects":8606,"num_object_clones":0,"num_object_copies":17212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10141,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2514},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12338","reported_seq":12476,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.613011+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:43.613011+0000","last_peered":"2024-09-05T13:17:43.613011+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:43.613011+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35110912,"num_objects":8572,"num_object_clones":0,"num_object_copies":17144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8705,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4118},{"shards":"0,2","objects":2575}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12373","reported_seq":21274,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.147886+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:43.147886+0000","last_peered":"2024-09-05T13:17:43.147886+0000","last_clean":"2024-09-05T13:17:43.147886+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:43.147886+0000","last_undegraded":"2024-09-05T13:17:43.147886+0000","last_fullsized":"2024-09-05T13:17:43.147886+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34971648,"num_objects":8538,"num_object_clones":0,"num_object_copies":17076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9968,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12157","reported_seq":12280,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.105725+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:43.105725+0000","last_peered":"2024-09-05T13:17:43.105725+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:43.105725+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34508800,"num_objects":8425,"num_object_clones":0,"num_object_copies":16850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9666,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2524},{"shards":"2","objects":4038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12367","reported_seq":12508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.587175+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:43.587175+0000","last_peered":"2024-09-05T13:17:43.587175+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:43.587175+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34861056,"num_objects":8511,"num_object_clones":0,"num_object_copies":17022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8518,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4092},{"shards":"0,2","objects":2492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12482","reported_seq":12670,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.578553+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:43.578553+0000","last_peered":"2024-09-05T13:17:43.578553+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:43.578553+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35315712,"num_objects":8622,"num_object_clones":0,"num_object_copies":17244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8381,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":2518}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12482","reported_seq":12621,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.535728+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:43.535728+0000","last_peered":"2024-09-05T13:17:43.535728+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:43.535728+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35217408,"num_objects":8598,"num_object_clones":0,"num_object_copies":17196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8612,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4032},{"shards":"0,2","objects":2626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12517","reported_seq":12657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.501278+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:43.501278+0000","last_peered":"2024-09-05T13:17:43.501278+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:43.501278+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35533283,"num_objects":8676,"num_object_clones":0,"num_object_copies":17352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9185,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4111},{"shards":"0,2","objects":2650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":561586683,"num_objects":137108,"num_object_clones":0,"num_object_copies":274216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44938,"num_objects_misplaced":2537,"num_objects_unfound":0,"num_objects_dirty":137108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154079,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6021,"num_bytes_recovered":24653824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1227579392,"data_stored":1227488149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701648,"kb_used_data":417276,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93670192,"statfs":{"total":96636764160,"available":95918276608,"internally_reserved":0,"allocated":427290624,"data_stored":427031067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":866740,"kb_used_data":483868,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93505100,"statfs":{"total":96636764160,"available":95749222400,"internally_reserved":0,"allocated":495480832,"data_stored":495221270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,4,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584312,"kb_used_data":299940,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93787528,"statfs":{"total":96636764160,"available":96038428672,"internally_reserved":0,"allocated":307138560,"data_stored":306802666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":493961216,"data_stored":493957603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306728960,"data_stored":306649015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426889216,"data_stored":426881531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:48.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:49.537 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:49.537 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:49.566 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-05T13:17:47.631175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":561586702,"num_objects":137110,"num_object_clones":0,"num_object_copies":274220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44938,"num_objects_misplaced":2537,"num_objects_unfound":0,"num_objects_dirty":137110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154081,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6022,"num_bytes_recovered":24653843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160964,"ondisk_log_size":160964,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2156288,"kb_used_data":1200516,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":280959232,"statfs":{"total":289910292480,"available":287702253568,"internally_reserved":0,"allocated":1229328384,"data_stored":1228473371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":978635782},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,2,10,12,9,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18116608,"num_objects":-4423,"num_object_clones":0,"num_object_copies":-8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":102,"num_objects_misplaced":-279,"num_objects_unfound":0,"num_objects_dirty":-4423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4423,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004295"},"pg_stats":[{"pgid":"2.f","version":"45'12367","reported_seq":12442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.528473+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:43.528473+0000","last_peered":"2024-09-05T13:17:43.528473+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:43.528473+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35246080,"num_objects":8605,"num_object_clones":0,"num_object_copies":17210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12367,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4184},{"shards":"0,2","objects":2539}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12250","reported_seq":12355,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.143077+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:43.143077+0000","last_peered":"2024-09-05T13:17:43.143077+0000","last_clean":"2024-09-05T13:17:43.143077+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:43.143077+0000","last_undegraded":"2024-09-05T13:17:43.143077+0000","last_fullsized":"2024-09-05T13:17:43.143077+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12569","reported_seq":12698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.204291+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:43.204291+0000","last_peered":"2024-09-05T13:17:43.204291+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:43.204291+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332120,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10380,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2523},{"shards":"2","objects":4135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12393","reported_seq":12463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.120958+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:43.120958+0000","last_peered":"2024-09-05T13:17:43.120958+0000","last_clean":"2024-09-05T13:17:43.120958+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:43.120958+0000","last_undegraded":"2024-09-05T13:17:43.120958+0000","last_fullsized":"2024-09-05T13:17:43.120958+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12393,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12355","reported_seq":12490,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.552929+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:43.552929+0000","last_peered":"2024-09-05T13:17:43.552929+0000","last_clean":"2024-09-05T13:17:43.552929+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:43.552929+0000","last_undegraded":"2024-09-05T13:17:43.552929+0000","last_fullsized":"2024-09-05T13:17:43.552929+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34930688,"num_objects":8528,"num_object_clones":0,"num_object_copies":17056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8895,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12494","reported_seq":12633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.602425+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:43.602425+0000","last_peered":"2024-09-05T13:17:43.602425+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:43.602425+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35475456,"num_objects":8661,"num_object_clones":0,"num_object_copies":17322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8949,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4210},{"shards":"0,2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12214","reported_seq":15613,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.587785+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:43.587785+0000","last_peered":"2024-09-05T13:17:43.587785+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:43.587785+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":270,"num_objects_misplaced":2537,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8533,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1629,"num_bytes_recovered":6672384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5607},{"shards":"0,2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12516","reported_seq":12648,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.534188+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:43.534188+0000","last_peered":"2024-09-05T13:17:43.534188+0000","last_clean":"2024-09-05T13:17:43.534188+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:43.534188+0000","last_undegraded":"2024-09-05T13:17:43.534188+0000","last_fullsized":"2024-09-05T13:17:43.534188+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35176448,"num_objects":8588,"num_object_clones":0,"num_object_copies":17176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9126,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12487","reported_seq":12622,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.080376+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:43.080376+0000","last_peered":"2024-09-05T13:17:43.080376+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:43.080376+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250176,"num_objects":8606,"num_object_clones":0,"num_object_copies":17212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10141,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2514},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12338","reported_seq":12476,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.613011+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:43.613011+0000","last_peered":"2024-09-05T13:17:43.613011+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:43.613011+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35110912,"num_objects":8572,"num_object_clones":0,"num_object_copies":17144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8705,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4118},{"shards":"0,2","objects":2575}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12373","reported_seq":21274,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:43.147886+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:43.147886+0000","last_peered":"2024-09-05T13:17:43.147886+0000","last_clean":"2024-09-05T13:17:43.147886+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:43.147886+0000","last_undegraded":"2024-09-05T13:17:43.147886+0000","last_fullsized":"2024-09-05T13:17:43.147886+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34971648,"num_objects":8538,"num_object_clones":0,"num_object_copies":17076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9968,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12157","reported_seq":12280,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.105725+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:43.105725+0000","last_peered":"2024-09-05T13:17:43.105725+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:43.105725+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34508800,"num_objects":8425,"num_object_clones":0,"num_object_copies":16850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9666,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2524},{"shards":"2","objects":4038}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12367","reported_seq":12508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.587175+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:43.587175+0000","last_peered":"2024-09-05T13:17:43.587175+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:43.587175+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34861056,"num_objects":8511,"num_object_clones":0,"num_object_copies":17022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8518,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4092},{"shards":"0,2","objects":2492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12482","reported_seq":12670,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.578553+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:43.578553+0000","last_peered":"2024-09-05T13:17:43.578553+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:43.578553+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35315712,"num_objects":8622,"num_object_clones":0,"num_object_copies":17244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8381,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":2518}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12482","reported_seq":12621,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.535728+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:43.535728+0000","last_peered":"2024-09-05T13:17:43.535728+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:43.535728+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35217408,"num_objects":8598,"num_object_clones":0,"num_object_copies":17196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8612,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4032},{"shards":"0,2","objects":2626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12517","reported_seq":12657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:43.501278+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:43.501278+0000","last_peered":"2024-09-05T13:17:43.501278+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:43.501278+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35533283,"num_objects":8676,"num_object_clones":0,"num_object_copies":17352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9185,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4111},{"shards":"0,2","objects":2650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":561586683,"num_objects":137108,"num_object_clones":0,"num_object_copies":274216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44938,"num_objects_misplaced":2537,"num_objects_unfound":0,"num_objects_dirty":137108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154079,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6021,"num_bytes_recovered":24653824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226993664,"data_stored":1226902421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701648,"kb_used_data":417276,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93670192,"statfs":{"total":96636764160,"available":95918276608,"internally_reserved":0,"allocated":427290624,"data_stored":427031067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":866740,"kb_used_data":483868,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93505100,"statfs":{"total":96636764160,"available":95749222400,"internally_reserved":0,"allocated":495480832,"data_stored":495221270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,4,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587900,"kb_used_data":299372,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93783940,"statfs":{"total":96636764160,"available":96034754560,"internally_reserved":0,"allocated":306556928,"data_stored":306221034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":493961216,"data_stored":493957603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306143232,"data_stored":306063287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426889216,"data_stored":426881531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:49.567 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:50.132 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:50.155 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-05T13:17:49.631967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":552563214,"num_objects":134907,"num_object_clones":0,"num_object_copies":269814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44977,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":134907,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156284,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6118,"num_bytes_recovered":25047059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2137132,"kb_used_data":1189688,"kb_used_omap":12,"kb_used_meta":947379,"kb_avail":280978388,"statfs":{"total":289910292480,"available":287721869312,"internally_reserved":0,"allocated":1218240512,"data_stored":1217385499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":970116102},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,2,2,10,6,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27140096,"num_objects":-6626,"num_object_clones":0,"num_object_copies":-13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":141,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":-6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004478"},"pg_stats":[{"pgid":"2.f","version":"45'12487","reported_seq":12562,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.602536+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:48.602536+0000","last_peered":"2024-09-05T13:17:48.602536+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:48.602536+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12487,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4064},{"shards":"0,2","objects":2419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12384","reported_seq":12489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.103183+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:48.103183+0000","last_peered":"2024-09-05T13:17:48.103183+0000","last_clean":"2024-09-05T13:17:48.103183+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:48.103183+0000","last_undegraded":"2024-09-05T13:17:48.103183+0000","last_fullsized":"2024-09-05T13:17:48.103183+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34283520,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10394,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12706","reported_seq":12835,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.182236+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:48.182236+0000","last_peered":"2024-09-05T13:17:48.182236+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:48.182236+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770968,"num_objects":8490,"num_object_clones":0,"num_object_copies":16980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10517,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2386},{"shards":"2","objects":3998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12522","reported_seq":12592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.136940+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:48.136940+0000","last_peered":"2024-09-05T13:17:48.136940+0000","last_clean":"2024-09-05T13:17:48.136940+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:48.136940+0000","last_undegraded":"2024-09-05T13:17:48.136940+0000","last_fullsized":"2024-09-05T13:17:48.136940+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12522,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12491","reported_seq":12626,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.650037+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:48.650037+0000","last_peered":"2024-09-05T13:17:48.650037+0000","last_clean":"2024-09-05T13:17:48.650037+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:48.650037+0000","last_undegraded":"2024-09-05T13:17:48.650037+0000","last_fullsized":"2024-09-05T13:17:48.650037+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9031,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12629","reported_seq":12768,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.672187+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:48.672187+0000","last_peered":"2024-09-05T13:17:48.672187+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:48.672187+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34922496,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9084,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4075},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12348","reported_seq":15939,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.660017+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:48.660017+0000","last_peered":"2024-09-05T13:17:48.660017+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:48.660017+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33914880,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":309,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1725,"num_bytes_recovered":7065600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5569},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12662","reported_seq":12794,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.650819+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:48.650819+0000","last_peered":"2024-09-05T13:17:48.650819+0000","last_clean":"2024-09-05T13:17:48.650819+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:48.650819+0000","last_undegraded":"2024-09-05T13:17:48.650819+0000","last_fullsized":"2024-09-05T13:17:48.650819+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9272,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12640","reported_seq":12771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.226302+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:48.226302+0000","last_peered":"2024-09-05T13:17:48.226302+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:48.226302+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34623488,"num_objects":8453,"num_object_clones":0,"num_object_copies":16906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10294,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2361},{"shards":"2","objects":4002}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12481","reported_seq":12619,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.669546+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:48.669546+0000","last_peered":"2024-09-05T13:17:48.669546+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:48.669546+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34525184,"num_objects":8429,"num_object_clones":0,"num_object_copies":16858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8848,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12512","reported_seq":21413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.140271+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:48.140271+0000","last_peered":"2024-09-05T13:17:48.140271+0000","last_clean":"2024-09-05T13:17:48.140271+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:48.140271+0000","last_undegraded":"2024-09-05T13:17:48.140271+0000","last_fullsized":"2024-09-05T13:17:48.140271+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10107,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12306","reported_seq":12429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.107711+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:48.107711+0000","last_peered":"2024-09-05T13:17:48.107711+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:48.107711+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33898496,"num_objects":8276,"num_object_clones":0,"num_object_copies":16552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9815,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2375},{"shards":"2","objects":3889}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12501","reported_seq":12642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.682908+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:48.682908+0000","last_peered":"2024-09-05T13:17:48.682908+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:48.682908+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34312192,"num_objects":8377,"num_object_clones":0,"num_object_copies":16754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8652,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3958},{"shards":"0,2","objects":2358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12606","reported_seq":12794,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.610705+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:48.610705+0000","last_peered":"2024-09-05T13:17:48.610705+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:48.610705+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34807808,"num_objects":8498,"num_object_clones":0,"num_object_copies":16996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8505,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4054},{"shards":"0,2","objects":2394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12632","reported_seq":12770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.685493+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:48.685493+0000","last_peered":"2024-09-05T13:17:48.685493+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:48.685493+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3882},{"shards":"0,2","objects":2476}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12657","reported_seq":12797,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.625278+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:48.625278+0000","last_peered":"2024-09-05T13:17:48.625278+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:48.625278+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9325,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":2510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":552563195,"num_objects":134905,"num_object_clones":0,"num_object_copies":269810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44977,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":134905,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156282,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6117,"num_bytes_recovered":25047040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1215733760,"data_stored":1215642517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701404,"kb_used_data":412872,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93670436,"statfs":{"total":96636764160,"available":95918526464,"internally_reserved":0,"allocated":422780928,"data_stored":422521371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847828,"kb_used_data":477444,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93524012,"statfs":{"total":96636764160,"available":95768588288,"internally_reserved":0,"allocated":488902656,"data_stored":488643094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,1,4,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587900,"kb_used_data":299372,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93783940,"statfs":{"total":96636764160,"available":96034754560,"internally_reserved":0,"allocated":306556928,"data_stored":306221034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487223296,"data_stored":487219683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306143232,"data_stored":306063287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422367232,"data_stored":422359547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:50.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:50.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12487", 'reported_seq': 12562, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.602536+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:48.602536+0000', 'last_peered': '2024-09-05T13:17:48.602536+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:48.602536+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34754560, 'num_objects': 8485, 'num_object_clones': 0, 'num_object_copies': 16970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12487, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4064}, {'shards': '0,2', 'objects': 2419}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:50.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:50.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12706", 'reported_seq': 12835, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.182236+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:48.182236+0000', 'last_peered': '2024-09-05T13:17:48.182236+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:48.182236+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34770968, 'num_objects': 8490, 'num_object_clones': 0, 'num_object_copies': 16980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8490, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10517, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2386}, {'shards': '2', 'objects': 3998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:50.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:50.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12629", 'reported_seq': 12768, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.672187+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:48.672187+0000', 'last_peered': '2024-09-05T13:17:48.672187+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:48.672187+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34922496, 'num_objects': 8526, 'num_object_clones': 0, 'num_object_copies': 17052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9084, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4075}, {'shards': '0,2', 'objects': 2402}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:50.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:50.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12348", 'reported_seq': 15939, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.660017+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:48.660017+0000', 'last_peered': '2024-09-05T13:17:48.660017+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:48.660017+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33914880, 'num_objects': 8280, 'num_object_clones': 0, 'num_object_copies': 16560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 309, 'num_objects_misplaced': 2402, 'num_objects_unfound': 0, 'num_objects_dirty': 8280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8667, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1725, 'num_bytes_recovered': 7065600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5569}, {'shards': '0,2', 'objects': 2402}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:50.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:50.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12640", 'reported_seq': 12771, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.226302+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:48.226302+0000', 'last_peered': '2024-09-05T13:17:48.226302+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:48.226302+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34623488, 'num_objects': 8453, 'num_object_clones': 0, 'num_object_copies': 16906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8453, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10294, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2361}, {'shards': '2', 'objects': 4002}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:50.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:50.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12481", 'reported_seq': 12619, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.669546+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:48.669546+0000', 'last_peered': '2024-09-05T13:17:48.669546+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:48.669546+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34525184, 'num_objects': 8429, 'num_object_clones': 0, 'num_object_copies': 16858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8848, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3975}, {'shards': '0,2', 'objects': 2432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:50.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:50.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'12306", 'reported_seq': 12429, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.107711+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:48.107711+0000', 'last_peered': '2024-09-05T13:17:48.107711+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:48.107711+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33898496, 'num_objects': 8276, 'num_object_clones': 0, 'num_object_copies': 16552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8276, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9815, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2375}, {'shards': '2', 'objects': 3889}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:50.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:50.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12501", 'reported_seq': 12642, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.682908+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:48.682908+0000', 'last_peered': '2024-09-05T13:17:48.682908+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:48.682908+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34312192, 'num_objects': 8377, 'num_object_clones': 0, 'num_object_copies': 16754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8652, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3958}, {'shards': '0,2', 'objects': 2358}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:50.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:50.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12606", 'reported_seq': 12794, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.610705+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:48.610705+0000', 'last_peered': '2024-09-05T13:17:48.610705+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:48.610705+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34807808, 'num_objects': 8498, 'num_object_clones': 0, 'num_object_copies': 16996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8505, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4054}, {'shards': '0,2', 'objects': 2394}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:50.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:50.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12632", 'reported_seq': 12770, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.685493+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:48.685493+0000', 'last_peered': '2024-09-05T13:17:48.685493+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:48.685493+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34603008, 'num_objects': 8448, 'num_object_clones': 0, 'num_object_copies': 16896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8762, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3882}, {'shards': '0,2', 'objects': 2476}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:50.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:50.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12657", 'reported_seq': 12797, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.625278+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:48.625278+0000', 'last_peered': '2024-09-05T13:17:48.625278+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:48.625278+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34959843, 'num_objects': 8536, 'num_object_clones': 0, 'num_object_copies': 17072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8536, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9325, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3971}, {'shards': '0,2', 'objects': 2510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:50.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:50.865 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:50.865 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44977/269814 objects degraded (16.670%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":397,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":134907,"data_bytes":552563214,"bytes_used":2188423168,"bytes_avail":287721869312,"bytes_total":289910292480,"degraded_objects":44977,"degraded_total":269814,"degraded_ratio":0.16669631672188989,"misplaced_objects":2402,"misplaced_total":269814,"misplaced_ratio":0.0089024290807741633,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93149,"recovering_keys_per_sec":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":551},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:50.887 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:50.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:51.447 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:51.469 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-05T13:17:49.631967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":552563214,"num_objects":134907,"num_object_clones":0,"num_object_copies":269814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44977,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":134907,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156284,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6118,"num_bytes_recovered":25047059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2137132,"kb_used_data":1189688,"kb_used_omap":12,"kb_used_meta":947379,"kb_avail":280978388,"statfs":{"total":289910292480,"available":287721869312,"internally_reserved":0,"allocated":1218240512,"data_stored":1217385499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":970116102},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,2,2,10,6,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27140096,"num_objects":-6626,"num_object_clones":0,"num_object_copies":-13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":141,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":-6626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6626,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004478"},"pg_stats":[{"pgid":"2.f","version":"45'12487","reported_seq":12562,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.602536+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:48.602536+0000","last_peered":"2024-09-05T13:17:48.602536+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:48.602536+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12487,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4064},{"shards":"0,2","objects":2419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12384","reported_seq":12489,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.103183+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:48.103183+0000","last_peered":"2024-09-05T13:17:48.103183+0000","last_clean":"2024-09-05T13:17:48.103183+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:48.103183+0000","last_undegraded":"2024-09-05T13:17:48.103183+0000","last_fullsized":"2024-09-05T13:17:48.103183+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34283520,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10394,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12706","reported_seq":12835,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.182236+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:48.182236+0000","last_peered":"2024-09-05T13:17:48.182236+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:48.182236+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770968,"num_objects":8490,"num_object_clones":0,"num_object_copies":16980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10517,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2386},{"shards":"2","objects":3998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12522","reported_seq":12592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.136940+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:48.136940+0000","last_peered":"2024-09-05T13:17:48.136940+0000","last_clean":"2024-09-05T13:17:48.136940+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:48.136940+0000","last_undegraded":"2024-09-05T13:17:48.136940+0000","last_fullsized":"2024-09-05T13:17:48.136940+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12522,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12491","reported_seq":12626,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.650037+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:48.650037+0000","last_peered":"2024-09-05T13:17:48.650037+0000","last_clean":"2024-09-05T13:17:48.650037+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:48.650037+0000","last_undegraded":"2024-09-05T13:17:48.650037+0000","last_fullsized":"2024-09-05T13:17:48.650037+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9031,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12629","reported_seq":12768,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.672187+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:48.672187+0000","last_peered":"2024-09-05T13:17:48.672187+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:48.672187+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34922496,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9084,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4075},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12348","reported_seq":15939,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.660017+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:48.660017+0000","last_peered":"2024-09-05T13:17:48.660017+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:48.660017+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33914880,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":309,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1725,"num_bytes_recovered":7065600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5569},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12662","reported_seq":12794,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.650819+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:48.650819+0000","last_peered":"2024-09-05T13:17:48.650819+0000","last_clean":"2024-09-05T13:17:48.650819+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:48.650819+0000","last_undegraded":"2024-09-05T13:17:48.650819+0000","last_fullsized":"2024-09-05T13:17:48.650819+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9272,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12640","reported_seq":12771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.226302+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:48.226302+0000","last_peered":"2024-09-05T13:17:48.226302+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:48.226302+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34623488,"num_objects":8453,"num_object_clones":0,"num_object_copies":16906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10294,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2361},{"shards":"2","objects":4002}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12481","reported_seq":12619,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.669546+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:48.669546+0000","last_peered":"2024-09-05T13:17:48.669546+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:48.669546+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34525184,"num_objects":8429,"num_object_clones":0,"num_object_copies":16858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8848,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12512","reported_seq":21413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.140271+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:48.140271+0000","last_peered":"2024-09-05T13:17:48.140271+0000","last_clean":"2024-09-05T13:17:48.140271+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:48.140271+0000","last_undegraded":"2024-09-05T13:17:48.140271+0000","last_fullsized":"2024-09-05T13:17:48.140271+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10107,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12306","reported_seq":12429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.107711+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:48.107711+0000","last_peered":"2024-09-05T13:17:48.107711+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:48.107711+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33898496,"num_objects":8276,"num_object_clones":0,"num_object_copies":16552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9815,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2375},{"shards":"2","objects":3889}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12501","reported_seq":12642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.682908+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:48.682908+0000","last_peered":"2024-09-05T13:17:48.682908+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:48.682908+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34312192,"num_objects":8377,"num_object_clones":0,"num_object_copies":16754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8652,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3958},{"shards":"0,2","objects":2358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12606","reported_seq":12794,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.610705+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:48.610705+0000","last_peered":"2024-09-05T13:17:48.610705+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:48.610705+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34807808,"num_objects":8498,"num_object_clones":0,"num_object_copies":16996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8505,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4054},{"shards":"0,2","objects":2394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12632","reported_seq":12770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.685493+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:48.685493+0000","last_peered":"2024-09-05T13:17:48.685493+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:48.685493+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3882},{"shards":"0,2","objects":2476}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12657","reported_seq":12797,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.625278+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:48.625278+0000","last_peered":"2024-09-05T13:17:48.625278+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:48.625278+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9325,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":2510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":552563195,"num_objects":134905,"num_object_clones":0,"num_object_copies":269810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44977,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":134905,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156282,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6117,"num_bytes_recovered":25047040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1215733760,"data_stored":1215642517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701404,"kb_used_data":412872,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93670436,"statfs":{"total":96636764160,"available":95918526464,"internally_reserved":0,"allocated":422780928,"data_stored":422521371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847828,"kb_used_data":477444,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93524012,"statfs":{"total":96636764160,"available":95768588288,"internally_reserved":0,"allocated":488902656,"data_stored":488643094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,1,4,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587900,"kb_used_data":299372,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93783940,"statfs":{"total":96636764160,"available":96034754560,"internally_reserved":0,"allocated":306556928,"data_stored":306221034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487223296,"data_stored":487219683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306143232,"data_stored":306063287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422367232,"data_stored":422359547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:54.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:55.043 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:55.070 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-05T13:17:53.633580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549278222,"num_objects":134105,"num_object_clones":0,"num_object_copies":268210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44977,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":134105,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157086,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6118,"num_bytes_recovered":25047059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2128148,"kb_used_data":1184868,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":280987372,"statfs":{"total":289910292480,"available":287731068928,"internally_reserved":0,"allocated":1213304832,"data_stored":1212457045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":965856262},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,4,6,3,7,16,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21450752,"num_objects":-5237,"num_object_clones":0,"num_object_copies":-10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":99,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":-5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5237,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":753664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004780"},"pg_stats":[{"pgid":"2.f","version":"45'12487","reported_seq":12562,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.602536+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:48.602536+0000","last_peered":"2024-09-05T13:17:48.602536+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:48.602536+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12487,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4064},{"shards":"0,2","objects":2419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12508","reported_seq":12613,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.240590+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:53.240590+0000","last_peered":"2024-09-05T13:17:53.240590+0000","last_clean":"2024-09-05T13:17:53.240590+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:53.240590+0000","last_undegraded":"2024-09-05T13:17:53.240590+0000","last_fullsized":"2024-09-05T13:17:53.240590+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33775616,"num_objects":8246,"num_object_clones":0,"num_object_copies":16492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10518,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12846","reported_seq":12975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.175499+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:53.175499+0000","last_peered":"2024-09-05T13:17:53.175499+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:53.175499+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34197528,"num_objects":8350,"num_object_clones":0,"num_object_copies":16700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10657,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2246},{"shards":"2","objects":3858}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12658","reported_seq":12728,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.230090+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:53.230090+0000","last_peered":"2024-09-05T13:17:53.230090+0000","last_clean":"2024-09-05T13:17:53.230090+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:53.230090+0000","last_undegraded":"2024-09-05T13:17:53.230090+0000","last_fullsized":"2024-09-05T13:17:53.230090+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275328,"num_objects":8368,"num_object_clones":0,"num_object_copies":16736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12491","reported_seq":12626,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.650037+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:48.650037+0000","last_peered":"2024-09-05T13:17:48.650037+0000","last_clean":"2024-09-05T13:17:48.650037+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:48.650037+0000","last_undegraded":"2024-09-05T13:17:48.650037+0000","last_fullsized":"2024-09-05T13:17:48.650037+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9031,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12629","reported_seq":12768,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.672187+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:48.672187+0000","last_peered":"2024-09-05T13:17:48.672187+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:48.672187+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34922496,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9084,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4075},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12348","reported_seq":15939,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.660017+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:48.660017+0000","last_peered":"2024-09-05T13:17:48.660017+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:48.660017+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33914880,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":309,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1725,"num_bytes_recovered":7065600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5569},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12662","reported_seq":12794,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.650819+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:48.650819+0000","last_peered":"2024-09-05T13:17:48.650819+0000","last_clean":"2024-09-05T13:17:48.650819+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:48.650819+0000","last_undegraded":"2024-09-05T13:17:48.650819+0000","last_fullsized":"2024-09-05T13:17:48.650819+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9272,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12781","reported_seq":12916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.222697+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:53.222697+0000","last_peered":"2024-09-05T13:17:53.222697+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:53.222697+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10435,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2220},{"shards":"2","objects":3861}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12481","reported_seq":12619,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.669546+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:48.669546+0000","last_peered":"2024-09-05T13:17:48.669546+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:48.669546+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34525184,"num_objects":8429,"num_object_clones":0,"num_object_copies":16858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8848,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12633","reported_seq":21534,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.059109+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:53.059109+0000","last_peered":"2024-09-05T13:17:53.059109+0000","last_clean":"2024-09-05T13:17:53.059109+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:53.059109+0000","last_undegraded":"2024-09-05T13:17:53.059109+0000","last_fullsized":"2024-09-05T13:17:53.059109+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33906688,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10228,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12446","reported_seq":12569,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.184916+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:53.184916+0000","last_peered":"2024-09-05T13:17:53.184916+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:53.184916+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9955,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2235},{"shards":"2","objects":3749}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12501","reported_seq":12642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.682908+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:48.682908+0000","last_peered":"2024-09-05T13:17:48.682908+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:48.682908+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34312192,"num_objects":8377,"num_object_clones":0,"num_object_copies":16754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8652,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3958},{"shards":"0,2","objects":2358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12606","reported_seq":12794,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.610705+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:48.610705+0000","last_peered":"2024-09-05T13:17:48.610705+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:48.610705+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34807808,"num_objects":8498,"num_object_clones":0,"num_object_copies":16996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8505,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4054},{"shards":"0,2","objects":2394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12632","reported_seq":12770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.685493+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:48.685493+0000","last_peered":"2024-09-05T13:17:48.685493+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:48.685493+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3882},{"shards":"0,2","objects":2476}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12657","reported_seq":12797,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.625278+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:48.625278+0000","last_peered":"2024-09-05T13:17:48.625278+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:48.625278+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9325,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":2510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549278203,"num_objects":134103,"num_object_clones":0,"num_object_copies":268206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44977,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":134103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157084,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6117,"num_bytes_recovered":25047040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1210789888,"data_stored":1210705871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701264,"kb_used_data":408576,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93670576,"statfs":{"total":96636764160,"available":95918669824,"internally_reserved":0,"allocated":418381824,"data_stored":418122267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,0,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847828,"kb_used_data":477444,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93524012,"statfs":{"total":96636764160,"available":95768588288,"internally_reserved":0,"allocated":488902656,"data_stored":488643094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,1,4,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579056,"kb_used_data":298848,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93792784,"statfs":{"total":96636764160,"available":96043810816,"internally_reserved":0,"allocated":306020352,"data_stored":305691684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487223296,"data_stored":487219683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305606656,"data_stored":305533937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417959936,"data_stored":417952251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:55.072 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:55.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:55.620 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:55.639 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-05T13:17:53.633580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549278222,"num_objects":134105,"num_object_clones":0,"num_object_copies":268210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44977,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":134105,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157086,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6118,"num_bytes_recovered":25047059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2128148,"kb_used_data":1184868,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":280987372,"statfs":{"total":289910292480,"available":287731068928,"internally_reserved":0,"allocated":1213304832,"data_stored":1212457045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":965856262},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,4,6,3,7,16,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21450752,"num_objects":-5237,"num_object_clones":0,"num_object_copies":-10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":99,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":-5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5237,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":753664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004780"},"pg_stats":[{"pgid":"2.f","version":"45'12487","reported_seq":12562,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.602536+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:48.602536+0000","last_peered":"2024-09-05T13:17:48.602536+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:48.602536+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12487,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4064},{"shards":"0,2","objects":2419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12508","reported_seq":12613,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.240590+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:53.240590+0000","last_peered":"2024-09-05T13:17:53.240590+0000","last_clean":"2024-09-05T13:17:53.240590+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:53.240590+0000","last_undegraded":"2024-09-05T13:17:53.240590+0000","last_fullsized":"2024-09-05T13:17:53.240590+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33775616,"num_objects":8246,"num_object_clones":0,"num_object_copies":16492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10518,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12846","reported_seq":12975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.175499+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:53.175499+0000","last_peered":"2024-09-05T13:17:53.175499+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:53.175499+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34197528,"num_objects":8350,"num_object_clones":0,"num_object_copies":16700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10657,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2246},{"shards":"2","objects":3858}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12658","reported_seq":12728,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.230090+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:53.230090+0000","last_peered":"2024-09-05T13:17:53.230090+0000","last_clean":"2024-09-05T13:17:53.230090+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:53.230090+0000","last_undegraded":"2024-09-05T13:17:53.230090+0000","last_fullsized":"2024-09-05T13:17:53.230090+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275328,"num_objects":8368,"num_object_clones":0,"num_object_copies":16736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12491","reported_seq":12626,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.650037+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:48.650037+0000","last_peered":"2024-09-05T13:17:48.650037+0000","last_clean":"2024-09-05T13:17:48.650037+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:48.650037+0000","last_undegraded":"2024-09-05T13:17:48.650037+0000","last_fullsized":"2024-09-05T13:17:48.650037+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34373632,"num_objects":8392,"num_object_clones":0,"num_object_copies":16784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9031,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12629","reported_seq":12768,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.672187+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:48.672187+0000","last_peered":"2024-09-05T13:17:48.672187+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:48.672187+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34922496,"num_objects":8526,"num_object_clones":0,"num_object_copies":17052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9084,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4075},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12348","reported_seq":15939,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.660017+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:48.660017+0000","last_peered":"2024-09-05T13:17:48.660017+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:48.660017+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33914880,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":309,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1725,"num_bytes_recovered":7065600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5569},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12662","reported_seq":12794,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:48.650819+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:48.650819+0000","last_peered":"2024-09-05T13:17:48.650819+0000","last_clean":"2024-09-05T13:17:48.650819+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:48.650819+0000","last_undegraded":"2024-09-05T13:17:48.650819+0000","last_fullsized":"2024-09-05T13:17:48.650819+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9272,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12781","reported_seq":12916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.222697+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:53.222697+0000","last_peered":"2024-09-05T13:17:53.222697+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:53.222697+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10435,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2220},{"shards":"2","objects":3861}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12481","reported_seq":12619,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.669546+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:48.669546+0000","last_peered":"2024-09-05T13:17:48.669546+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:48.669546+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34525184,"num_objects":8429,"num_object_clones":0,"num_object_copies":16858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8848,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3975},{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12633","reported_seq":21534,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.059109+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:53.059109+0000","last_peered":"2024-09-05T13:17:53.059109+0000","last_clean":"2024-09-05T13:17:53.059109+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:53.059109+0000","last_undegraded":"2024-09-05T13:17:53.059109+0000","last_fullsized":"2024-09-05T13:17:53.059109+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33906688,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10228,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12446","reported_seq":12569,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.184916+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:53.184916+0000","last_peered":"2024-09-05T13:17:53.184916+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:53.184916+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9955,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2235},{"shards":"2","objects":3749}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12501","reported_seq":12642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.682908+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:48.682908+0000","last_peered":"2024-09-05T13:17:48.682908+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:48.682908+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34312192,"num_objects":8377,"num_object_clones":0,"num_object_copies":16754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8652,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3958},{"shards":"0,2","objects":2358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12606","reported_seq":12794,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.610705+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:48.610705+0000","last_peered":"2024-09-05T13:17:48.610705+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:48.610705+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34807808,"num_objects":8498,"num_object_clones":0,"num_object_copies":16996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8505,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4054},{"shards":"0,2","objects":2394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12632","reported_seq":12770,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.685493+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:48.685493+0000","last_peered":"2024-09-05T13:17:48.685493+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:48.685493+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3882},{"shards":"0,2","objects":2476}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12657","reported_seq":12797,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:48.625278+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:48.625278+0000","last_peered":"2024-09-05T13:17:48.625278+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:48.625278+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9325,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":2510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549278203,"num_objects":134103,"num_object_clones":0,"num_object_copies":268206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44977,"num_objects_misplaced":2402,"num_objects_unfound":0,"num_objects_dirty":134103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157084,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6117,"num_bytes_recovered":25047040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1210789888,"data_stored":1210705871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701264,"kb_used_data":408576,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93670576,"statfs":{"total":96636764160,"available":95918669824,"internally_reserved":0,"allocated":418381824,"data_stored":418122267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,0,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847828,"kb_used_data":477444,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93524012,"statfs":{"total":96636764160,"available":95768588288,"internally_reserved":0,"allocated":488902656,"data_stored":488643094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,1,4,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579056,"kb_used_data":298848,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93792784,"statfs":{"total":96636764160,"available":96043810816,"internally_reserved":0,"allocated":306020352,"data_stored":305691684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487223296,"data_stored":487219683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305606656,"data_stored":305533937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417959936,"data_stored":417952251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:55.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:17:55.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12487", 'reported_seq': 12562, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.602536+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:48.602536+0000', 'last_peered': '2024-09-05T13:17:48.602536+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:48.602536+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34754560, 'num_objects': 8485, 'num_object_clones': 0, 'num_object_copies': 16970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12487, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4064}, {'shards': '0,2', 'objects': 2419}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:55.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:17:55.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12846", 'reported_seq': 12975, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:53.175499+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:53.175499+0000', 'last_peered': '2024-09-05T13:17:53.175499+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:53.175499+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34197528, 'num_objects': 8350, 'num_object_clones': 0, 'num_object_copies': 16700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8350, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10657, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2246}, {'shards': '2', 'objects': 3858}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:55.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:17:55.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12629", 'reported_seq': 12768, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.672187+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:48.672187+0000', 'last_peered': '2024-09-05T13:17:48.672187+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:48.672187+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34922496, 'num_objects': 8526, 'num_object_clones': 0, 'num_object_copies': 17052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9084, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4075}, {'shards': '0,2', 'objects': 2402}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:55.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:17:55.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12348", 'reported_seq': 15939, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.660017+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:48.660017+0000', 'last_peered': '2024-09-05T13:17:48.660017+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:48.660017+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33914880, 'num_objects': 8280, 'num_object_clones': 0, 'num_object_copies': 16560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 309, 'num_objects_misplaced': 2402, 'num_objects_unfound': 0, 'num_objects_dirty': 8280, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8667, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1725, 'num_bytes_recovered': 7065600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5569}, {'shards': '0,2', 'objects': 2402}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:55.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:17:55.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12781", 'reported_seq': 12916, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:53.222697+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:53.222697+0000', 'last_peered': '2024-09-05T13:17:53.222697+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:53.222697+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34045952, 'num_objects': 8312, 'num_object_clones': 0, 'num_object_copies': 16624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10435, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2220}, {'shards': '2', 'objects': 3861}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:55.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:17:55.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12481", 'reported_seq': 12619, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.669546+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:48.669546+0000', 'last_peered': '2024-09-05T13:17:48.669546+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:48.669546+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34525184, 'num_objects': 8429, 'num_object_clones': 0, 'num_object_copies': 16858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8848, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3975}, {'shards': '0,2', 'objects': 2432}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:55.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:17:55.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'12446", 'reported_seq': 12569, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:53.184916+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:53.184916+0000', 'last_peered': '2024-09-05T13:17:53.184916+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:53.184916+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33325056, 'num_objects': 8136, 'num_object_clones': 0, 'num_object_copies': 16272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9955, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2235}, {'shards': '2', 'objects': 3749}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:17:55.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:17:55.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12501", 'reported_seq': 12642, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.682908+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:48.682908+0000', 'last_peered': '2024-09-05T13:17:48.682908+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:48.682908+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34312192, 'num_objects': 8377, 'num_object_clones': 0, 'num_object_copies': 16754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8652, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3958}, {'shards': '0,2', 'objects': 2358}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:55.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:17:55.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12606", 'reported_seq': 12794, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.610705+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:48.610705+0000', 'last_peered': '2024-09-05T13:17:48.610705+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:48.610705+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34807808, 'num_objects': 8498, 'num_object_clones': 0, 'num_object_copies': 16996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8505, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4054}, {'shards': '0,2', 'objects': 2394}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:55.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:17:55.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12632", 'reported_seq': 12770, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.685493+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:48.685493+0000', 'last_peered': '2024-09-05T13:17:48.685493+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:48.685493+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34603008, 'num_objects': 8448, 'num_object_clones': 0, 'num_object_copies': 16896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8762, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3882}, {'shards': '0,2', 'objects': 2476}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:55.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:17:55.643 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12657", 'reported_seq': 12797, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:48.625278+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:48.625278+0000', 'last_peered': '2024-09-05T13:17:48.625278+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:48.625278+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34959843, 'num_objects': 8536, 'num_object_clones': 0, 'num_object_copies': 17072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8536, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9325, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3971}, {'shards': '0,2', 'objects': 2510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:17:55.644 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:17:56.337 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:56.337 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45024/265448 objects degraded (16.962%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":403,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":132724,"data_bytes":543621646,"bytes_used":2176757760,"bytes_avail":287733534720,"bytes_total":289910292480,"degraded_objects":45024,"degraded_total":265448,"degraded_ratio":0.16961514119526236,"misplaced_objects":2262,"misplaced_total":265448,"misplaced_ratio":0.0085214429944847955,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":94509,"recovering_keys_per_sec":0,"num_objects_recovered":279,"num_bytes_recovered":1134592,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:17:56.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:17:56.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:17:56.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:17:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:17:56.953 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-05T13:17:55.634476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":543621646,"num_objects":132724,"num_object_clones":0,"num_object_copies":265448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45024,"num_objects_misplaced":2262,"num_objects_unfound":0,"num_objects_dirty":132724,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158467,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6213,"num_bytes_recovered":25427987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160650,"ondisk_log_size":160650,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125740,"kb_used_data":1178300,"kb_used_omap":12,"kb_used_meta":947379,"kb_avail":280989780,"statfs":{"total":289910292480,"available":287733534720,"internally_reserved":0,"allocated":1206579200,"data_stored":1205731413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":970116102},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,9,3,10,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23805952,"num_objects":-5812,"num_object_clones":0,"num_object_copies":-11624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":146,"num_objects_misplaced":-423,"num_objects_unfound":0,"num_objects_dirty":-5812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5812,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005064"},"pg_stats":[{"pgid":"2.f","version":"45'12630","reported_seq":12705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.630183+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:53.630183+0000","last_peered":"2024-09-05T13:17:53.630183+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:53.630183+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34168832,"num_objects":8342,"num_object_clones":0,"num_object_copies":16684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12630,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":2276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12508","reported_seq":12613,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.240590+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:53.240590+0000","last_peered":"2024-09-05T13:17:53.240590+0000","last_clean":"2024-09-05T13:17:53.240590+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:53.240590+0000","last_undegraded":"2024-09-05T13:17:53.240590+0000","last_fullsized":"2024-09-05T13:17:53.240590+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33775616,"num_objects":8246,"num_object_clones":0,"num_object_copies":16492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10518,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12846","reported_seq":12975,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.175499+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:53.175499+0000","last_peered":"2024-09-05T13:17:53.175499+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:53.175499+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34197528,"num_objects":8350,"num_object_clones":0,"num_object_copies":16700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10657,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2246},{"shards":"2","objects":3858}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12658","reported_seq":12728,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.230090+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:53.230090+0000","last_peered":"2024-09-05T13:17:53.230090+0000","last_clean":"2024-09-05T13:17:53.230090+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:53.230090+0000","last_undegraded":"2024-09-05T13:17:53.230090+0000","last_fullsized":"2024-09-05T13:17:53.230090+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34275328,"num_objects":8368,"num_object_clones":0,"num_object_copies":16736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12610","reported_seq":12745,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.564567+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:53.564567+0000","last_peered":"2024-09-05T13:17:53.564567+0000","last_clean":"2024-09-05T13:17:53.564567+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:53.564567+0000","last_undegraded":"2024-09-05T13:17:53.564567+0000","last_fullsized":"2024-09-05T13:17:53.564567+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33886208,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9150,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12768","reported_seq":12907,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.690911+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:53.690911+0000","last_peered":"2024-09-05T13:17:53.690911+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:53.690911+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34353152,"num_objects":8387,"num_object_clones":0,"num_object_copies":16774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9223,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3936},{"shards":"0,2","objects":2263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12489","reported_seq":16270,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.730310+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:53.730310+0000","last_peered":"2024-09-05T13:17:53.730310+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:53.730310+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33337344,"num_objects":8139,"num_object_clones":0,"num_object_copies":16278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":356,"num_objects_misplaced":2262,"num_objects_unfound":0,"num_objects_dirty":8139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8808,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1820,"num_bytes_recovered":7446528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5521},{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12801","reported_seq":12933,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.712470+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:53.712470+0000","last_peered":"2024-09-05T13:17:53.712470+0000","last_clean":"2024-09-05T13:17:53.712470+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:53.712470+0000","last_undegraded":"2024-09-05T13:17:53.712470+0000","last_fullsized":"2024-09-05T13:17:53.712470+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34009088,"num_objects":8303,"num_object_clones":0,"num_object_copies":16606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9411,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12781","reported_seq":12916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.222697+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:53.222697+0000","last_peered":"2024-09-05T13:17:53.222697+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:53.222697+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10435,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2220},{"shards":"2","objects":3861}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12607","reported_seq":12745,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.694694+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:53.694694+0000","last_peered":"2024-09-05T13:17:53.694694+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:53.694694+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34009088,"num_objects":8303,"num_object_clones":0,"num_object_copies":16606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8974,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3849},{"shards":"0,2","objects":2306}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12633","reported_seq":21534,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:53.059109+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:53.059109+0000","last_peered":"2024-09-05T13:17:53.059109+0000","last_clean":"2024-09-05T13:17:53.059109+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:53.059109+0000","last_undegraded":"2024-09-05T13:17:53.059109+0000","last_fullsized":"2024-09-05T13:17:53.059109+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33906688,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10228,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12446","reported_seq":12569,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.184916+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:53.184916+0000","last_peered":"2024-09-05T13:17:53.184916+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:53.184916+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9955,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2235},{"shards":"2","objects":3749}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12630","reported_seq":12771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.615552+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:53.615552+0000","last_peered":"2024-09-05T13:17:53.615552+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:53.615552+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33783808,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8781,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3829},{"shards":"0,2","objects":2229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12749","reported_seq":12937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.757018+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:53.757018+0000","last_peered":"2024-09-05T13:17:53.757018+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:53.757018+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34222080,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8648,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3911},{"shards":"0,2","objects":2251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12776","reported_seq":12913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.653949+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:53.653949+0000","last_peered":"2024-09-05T13:17:53.653949+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:53.653949+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34013184,"num_objects":8304,"num_object_clones":0,"num_object_copies":16608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3738},{"shards":"0,2","objects":2332}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12815","reported_seq":12955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:53.767165+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:53.767165+0000","last_peered":"2024-09-05T13:17:53.767165+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:53.767165+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34312675,"num_objects":8378,"num_object_clones":0,"num_object_copies":16756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9483,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3813},{"shards":"0,2","objects":2352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":543621627,"num_objects":132722,"num_object_clones":0,"num_object_copies":265444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45024,"num_objects_misplaced":2262,"num_objects_unfound":0,"num_objects_dirty":132722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158465,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6212,"num_bytes_recovered":25427968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204072448,"data_stored":1203988431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160647,"ondisk_log_size":160647,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701264,"kb_used_data":408576,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93670576,"statfs":{"total":96636764160,"available":95918669824,"internally_reserved":0,"allocated":418381824,"data_stored":418122267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":299691693},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,0,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845420,"kb_used_data":470876,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93526420,"statfs":{"total":96636764160,"available":95771054080,"internally_reserved":0,"allocated":482177024,"data_stored":481917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579056,"kb_used_data":298848,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93792784,"statfs":{"total":96636764160,"available":96043810816,"internally_reserved":0,"allocated":306020352,"data_stored":305691684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":480505856,"data_stored":480502243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305606656,"data_stored":305533937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417959936,"data_stored":417952251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:17:59.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:00.530 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:00.531 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:00.556 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-05T13:17:59.636199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":534503950,"num_objects":130498,"num_object_clones":0,"num_object_copies":260996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45065,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":130498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160693,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6307,"num_bytes_recovered":25813011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2130848,"kb_used_data":1166744,"kb_used_omap":12,"kb_used_meta":964019,"kb_avail":280984672,"statfs":{"total":289910292480,"available":287728304128,"internally_reserved":0,"allocated":1194745856,"data_stored":1193898069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":987155462},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,6,5,14,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27082752,"num_objects":-6612,"num_object_clones":0,"num_object_copies":-13224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":127,"num_objects_misplaced":-410,"num_objects_unfound":0,"num_objects_dirty":-6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6612,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005023"},"pg_stats":[{"pgid":"2.f","version":"45'12748","reported_seq":12823,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.763055+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:58.763055+0000","last_peered":"2024-09-05T13:17:58.763055+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:58.763055+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33685504,"num_objects":8224,"num_object_clones":0,"num_object_copies":16448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12748,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3803},{"shards":"0,2","objects":2158}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12646","reported_seq":12750,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.293755+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:58.293755+0000","last_peered":"2024-09-05T13:17:58.293755+0000","last_clean":"2024-09-05T13:17:58.293755+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:58.293755+0000","last_undegraded":"2024-09-05T13:17:58.293755+0000","last_fullsized":"2024-09-05T13:17:58.293755+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33210368,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10656,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13001","reported_seq":13130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.274994+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:58.274994+0000","last_peered":"2024-09-05T13:17:58.274994+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:58.274994+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33562648,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10812,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2091},{"shards":"2","objects":3703}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12804","reported_seq":12874,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.172109+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:58.172109+0000","last_peered":"2024-09-05T13:17:58.172109+0000","last_clean":"2024-09-05T13:17:58.172109+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:58.172109+0000","last_undegraded":"2024-09-05T13:17:58.172109+0000","last_fullsized":"2024-09-05T13:17:58.172109+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33677312,"num_objects":8222,"num_object_clones":0,"num_object_copies":16444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12804,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12768","reported_seq":12903,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.705705+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:58.705705+0000","last_peered":"2024-09-05T13:17:58.705705+0000","last_clean":"2024-09-05T13:17:58.705705+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:58.705705+0000","last_undegraded":"2024-09-05T13:17:58.705705+0000","last_fullsized":"2024-09-05T13:17:58.705705+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12907","reported_seq":13045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.757669+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:58.757669+0000","last_peered":"2024-09-05T13:17:58.757669+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:58.757669+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33783808,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9362,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":2124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12625","reported_seq":16594,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.784016+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:58.784016+0000","last_peered":"2024-09-05T13:17:58.784016+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:58.784016+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":397,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8944,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1914,"num_bytes_recovered":7831552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5479},{"shards":"0,2","objects":2127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12939","reported_seq":13071,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.695373+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:58.695373+0000","last_peered":"2024-09-05T13:17:58.695373+0000","last_clean":"2024-09-05T13:17:58.695373+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:58.695373+0000","last_undegraded":"2024-09-05T13:17:58.695373+0000","last_fullsized":"2024-09-05T13:17:58.695373+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33443840,"num_objects":8165,"num_object_clones":0,"num_object_copies":16330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12890","reported_seq":13025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.198785+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:58.198785+0000","last_peered":"2024-09-05T13:17:58.198785+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:58.198785+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10544,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2111},{"shards":"2","objects":3752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12754","reported_seq":12892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.729399+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:58.729399+0000","last_peered":"2024-09-05T13:17:58.729399+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:58.729399+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9121,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3702},{"shards":"0,2","objects":2159}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12777","reported_seq":21678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.175126+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:58.175126+0000","last_peered":"2024-09-05T13:17:58.175126+0000","last_clean":"2024-09-05T13:17:58.175126+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:58.175126+0000","last_undegraded":"2024-09-05T13:17:58.175126+0000","last_fullsized":"2024-09-05T13:17:58.175126+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33316864,"num_objects":8134,"num_object_clones":0,"num_object_copies":16268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10372,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12584","reported_seq":12707,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.267148+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:58.267148+0000","last_peered":"2024-09-05T13:17:58.267148+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:58.267148+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32759808,"num_objects":7998,"num_object_clones":0,"num_object_copies":15996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10093,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2097},{"shards":"2","objects":3611}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12775","reported_seq":12916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.658757+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:58.658757+0000","last_peered":"2024-09-05T13:17:58.658757+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:58.658757+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8926,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3684},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12875","reported_seq":13063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.722979+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:58.722979+0000","last_peered":"2024-09-05T13:17:58.722979+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:58.722979+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8774,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12907","reported_seq":13045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.628106+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:58.628106+0000","last_peered":"2024-09-05T13:17:58.628106+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:58.628106+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33476608,"num_objects":8173,"num_object_clones":0,"num_object_copies":16346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9037,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12973","reported_seq":13113,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.760624+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:58.760624+0000","last_peered":"2024-09-05T13:17:58.760624+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:58.760624+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665507,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9641,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":534503931,"num_objects":130496,"num_object_clones":0,"num_object_copies":260992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45065,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":130496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160691,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6306,"num_bytes_recovered":25812992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1192124416,"data_stored":1192040399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700960,"kb_used_data":404112,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93670880,"statfs":{"total":96636764160,"available":95918981120,"internally_reserved":0,"allocated":413810688,"data_stored":413551131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":303951533},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847208,"kb_used_data":464320,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93524632,"statfs":{"total":96636764160,"available":95769223168,"internally_reserved":0,"allocated":475463680,"data_stored":475204118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582680,"kb_used_data":298312,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93789160,"statfs":{"total":96636764160,"available":96040099840,"internally_reserved":0,"allocated":305471488,"data_stored":305142820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473686016,"data_stored":473682403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305045504,"data_stored":304972785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413392896,"data_stored":413385211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:00.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:01.102 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:01.125 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-05T13:17:59.636199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":534503950,"num_objects":130498,"num_object_clones":0,"num_object_copies":260996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45065,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":130498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160693,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6307,"num_bytes_recovered":25813011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2130848,"kb_used_data":1166744,"kb_used_omap":12,"kb_used_meta":964019,"kb_avail":280984672,"statfs":{"total":289910292480,"available":287728304128,"internally_reserved":0,"allocated":1194745856,"data_stored":1193898069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":987155462},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,6,5,14,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27082752,"num_objects":-6612,"num_object_clones":0,"num_object_copies":-13224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":127,"num_objects_misplaced":-410,"num_objects_unfound":0,"num_objects_dirty":-6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6612,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005023"},"pg_stats":[{"pgid":"2.f","version":"45'12748","reported_seq":12823,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.763055+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:58.763055+0000","last_peered":"2024-09-05T13:17:58.763055+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:58.763055+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33685504,"num_objects":8224,"num_object_clones":0,"num_object_copies":16448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12748,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3803},{"shards":"0,2","objects":2158}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12646","reported_seq":12750,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.293755+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:58.293755+0000","last_peered":"2024-09-05T13:17:58.293755+0000","last_clean":"2024-09-05T13:17:58.293755+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:58.293755+0000","last_undegraded":"2024-09-05T13:17:58.293755+0000","last_fullsized":"2024-09-05T13:17:58.293755+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33210368,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10656,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13001","reported_seq":13130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.274994+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:58.274994+0000","last_peered":"2024-09-05T13:17:58.274994+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:58.274994+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33562648,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10812,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2091},{"shards":"2","objects":3703}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12804","reported_seq":12874,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.172109+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:58.172109+0000","last_peered":"2024-09-05T13:17:58.172109+0000","last_clean":"2024-09-05T13:17:58.172109+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:58.172109+0000","last_undegraded":"2024-09-05T13:17:58.172109+0000","last_fullsized":"2024-09-05T13:17:58.172109+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33677312,"num_objects":8222,"num_object_clones":0,"num_object_copies":16444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12804,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12768","reported_seq":12903,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.705705+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:58.705705+0000","last_peered":"2024-09-05T13:17:58.705705+0000","last_clean":"2024-09-05T13:17:58.705705+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:58.705705+0000","last_undegraded":"2024-09-05T13:17:58.705705+0000","last_fullsized":"2024-09-05T13:17:58.705705+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12907","reported_seq":13045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.757669+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:58.757669+0000","last_peered":"2024-09-05T13:17:58.757669+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:58.757669+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33783808,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9362,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":2124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12625","reported_seq":16594,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.784016+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:58.784016+0000","last_peered":"2024-09-05T13:17:58.784016+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:58.784016+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":397,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8944,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1914,"num_bytes_recovered":7831552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5479},{"shards":"0,2","objects":2127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12939","reported_seq":13071,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.695373+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:58.695373+0000","last_peered":"2024-09-05T13:17:58.695373+0000","last_clean":"2024-09-05T13:17:58.695373+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:58.695373+0000","last_undegraded":"2024-09-05T13:17:58.695373+0000","last_fullsized":"2024-09-05T13:17:58.695373+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33443840,"num_objects":8165,"num_object_clones":0,"num_object_copies":16330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12890","reported_seq":13025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.198785+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:58.198785+0000","last_peered":"2024-09-05T13:17:58.198785+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:58.198785+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10544,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2111},{"shards":"2","objects":3752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12754","reported_seq":12892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.729399+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:58.729399+0000","last_peered":"2024-09-05T13:17:58.729399+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:58.729399+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9121,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3702},{"shards":"0,2","objects":2159}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12777","reported_seq":21678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.175126+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:58.175126+0000","last_peered":"2024-09-05T13:17:58.175126+0000","last_clean":"2024-09-05T13:17:58.175126+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:58.175126+0000","last_undegraded":"2024-09-05T13:17:58.175126+0000","last_fullsized":"2024-09-05T13:17:58.175126+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33316864,"num_objects":8134,"num_object_clones":0,"num_object_copies":16268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10372,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12584","reported_seq":12707,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.267148+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:58.267148+0000","last_peered":"2024-09-05T13:17:58.267148+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:58.267148+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32759808,"num_objects":7998,"num_object_clones":0,"num_object_copies":15996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10093,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2097},{"shards":"2","objects":3611}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12775","reported_seq":12916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.658757+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:58.658757+0000","last_peered":"2024-09-05T13:17:58.658757+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:58.658757+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8926,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3684},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12875","reported_seq":13063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.722979+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:58.722979+0000","last_peered":"2024-09-05T13:17:58.722979+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:58.722979+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8774,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12907","reported_seq":13045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.628106+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:58.628106+0000","last_peered":"2024-09-05T13:17:58.628106+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:58.628106+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33476608,"num_objects":8173,"num_object_clones":0,"num_object_copies":16346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9037,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12973","reported_seq":13113,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.760624+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:58.760624+0000","last_peered":"2024-09-05T13:17:58.760624+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:58.760624+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665507,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9641,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":534503931,"num_objects":130496,"num_object_clones":0,"num_object_copies":260992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45065,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":130496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160691,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6306,"num_bytes_recovered":25812992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1192124416,"data_stored":1192040399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700960,"kb_used_data":404112,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93670880,"statfs":{"total":96636764160,"available":95918981120,"internally_reserved":0,"allocated":413810688,"data_stored":413551131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":303951533},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847208,"kb_used_data":464320,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93524632,"statfs":{"total":96636764160,"available":95769223168,"internally_reserved":0,"allocated":475463680,"data_stored":475204118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582680,"kb_used_data":298312,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93789160,"statfs":{"total":96636764160,"available":96040099840,"internally_reserved":0,"allocated":305471488,"data_stored":305142820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473686016,"data_stored":473682403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305045504,"data_stored":304972785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413392896,"data_stored":413385211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:01.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:01.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12748", 'reported_seq': 12823, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.763055+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:17:58.763055+0000', 'last_peered': '2024-09-05T13:17:58.763055+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:17:58.763055+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33685504, 'num_objects': 8224, 'num_object_clones': 0, 'num_object_copies': 16448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12748, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3803}, {'shards': '0,2', 'objects': 2158}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:01.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:01.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'13001", 'reported_seq': 13130, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.274994+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:17:58.274994+0000', 'last_peered': '2024-09-05T13:17:58.274994+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:17:58.274994+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33562648, 'num_objects': 8195, 'num_object_clones': 0, 'num_object_copies': 16390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8195, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10812, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2091}, {'shards': '2', 'objects': 3703}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:01.127 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:01.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12907", 'reported_seq': 13045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.757669+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:17:58.757669+0000', 'last_peered': '2024-09-05T13:17:58.757669+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:17:58.757669+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33783808, 'num_objects': 8248, 'num_object_clones': 0, 'num_object_copies': 16496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8248, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9362, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3797}, {'shards': '0,2', 'objects': 2124}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:01.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:01.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12625", 'reported_seq': 16594, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.784016+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:17:58.784016+0000', 'last_peered': '2024-09-05T13:17:58.784016+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:17:58.784016+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32780288, 'num_objects': 8003, 'num_object_clones': 0, 'num_object_copies': 16006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 397, 'num_objects_misplaced': 2127, 'num_objects_unfound': 0, 'num_objects_dirty': 8003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8944, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1914, 'num_bytes_recovered': 7831552, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5479}, {'shards': '0,2', 'objects': 2127}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:01.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:01.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12890", 'reported_seq': 13025, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.198785+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:17:58.198785+0000', 'last_peered': '2024-09-05T13:17:58.198785+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:17:58.198785+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33599488, 'num_objects': 8203, 'num_object_clones': 0, 'num_object_copies': 16406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10544, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2111}, {'shards': '2', 'objects': 3752}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:01.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:01.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12754", 'reported_seq': 12892, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.729399+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:17:58.729399+0000', 'last_peered': '2024-09-05T13:17:58.729399+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:17:58.729399+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33406976, 'num_objects': 8156, 'num_object_clones': 0, 'num_object_copies': 16312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9121, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3702}, {'shards': '0,2', 'objects': 2159}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:01.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:01.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'12584", 'reported_seq': 12707, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.267148+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:17:58.267148+0000', 'last_peered': '2024-09-05T13:17:58.267148+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:17:58.267148+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32759808, 'num_objects': 7998, 'num_object_clones': 0, 'num_object_copies': 15996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7998, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10093, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2097}, {'shards': '2', 'objects': 3611}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:01.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:01.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12775", 'reported_seq': 12916, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.658757+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:17:58.658757+0000', 'last_peered': '2024-09-05T13:17:58.658757+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:17:58.658757+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33189888, 'num_objects': 8103, 'num_object_clones': 0, 'num_object_copies': 16206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8926, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3684}, {'shards': '0,2', 'objects': 2084}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:01.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:01.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12875", 'reported_seq': 13063, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.722979+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:17:58.722979+0000', 'last_peered': '2024-09-05T13:17:58.722979+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:17:58.722979+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33705984, 'num_objects': 8229, 'num_object_clones': 0, 'num_object_copies': 16458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8774, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3785}, {'shards': '0,2', 'objects': 2125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:01.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:01.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12907", 'reported_seq': 13045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.628106+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:17:58.628106+0000', 'last_peered': '2024-09-05T13:17:58.628106+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:17:58.628106+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33476608, 'num_objects': 8173, 'num_object_clones': 0, 'num_object_copies': 16346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9037, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3607}, {'shards': '0,2', 'objects': 2201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:01.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:01.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12973", 'reported_seq': 13113, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:17:58.760624+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:17:58.760624+0000', 'last_peered': '2024-09-05T13:17:58.760624+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:17:58.760624+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33665507, 'num_objects': 8220, 'num_object_clones': 0, 'num_object_copies': 16440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9641, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3655}, {'shards': '0,2', 'objects': 2194}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:01.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:01.798 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:01.799 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45065/260996 objects degraded (17.267%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":408,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":130498,"data_bytes":534503950,"bytes_used":2181988352,"bytes_avail":287728304128,"bytes_total":289910292480,"degraded_objects":45065,"degraded_total":260996,"degraded_ratio":0.17266548146331742,"misplaced_objects":2127,"misplaced_total":260996,"misplaced_ratio":0.0081495501846771592,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96556,"recovering_keys_per_sec":0,"num_objects_recovered":285,"num_bytes_recovered":1159168,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":550},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:01.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:01.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:02.404 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:02.432 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-05T13:18:01.637045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":534503950,"num_objects":130498,"num_object_clones":0,"num_object_copies":260996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45065,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":130498,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160693,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6307,"num_bytes_recovered":25813011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2130848,"kb_used_data":1166744,"kb_used_omap":12,"kb_used_meta":964019,"kb_avail":280984672,"statfs":{"total":289910292480,"available":287728304128,"internally_reserved":0,"allocated":1194745856,"data_stored":1193898069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":987155462},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,6,5,14,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18059264,"num_objects":-4409,"num_object_clones":0,"num_object_copies":-8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":88,"num_objects_misplaced":-275,"num_objects_unfound":0,"num_objects_dirty":-4409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4409,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":765952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005078"},"pg_stats":[{"pgid":"2.f","version":"45'12748","reported_seq":12823,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.763055+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:17:58.763055+0000","last_peered":"2024-09-05T13:17:58.763055+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:17:58.763055+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33685504,"num_objects":8224,"num_object_clones":0,"num_object_copies":16448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12748,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3803},{"shards":"0,2","objects":2158}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12646","reported_seq":12750,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.293755+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:17:58.293755+0000","last_peered":"2024-09-05T13:17:58.293755+0000","last_clean":"2024-09-05T13:17:58.293755+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:17:58.293755+0000","last_undegraded":"2024-09-05T13:17:58.293755+0000","last_fullsized":"2024-09-05T13:17:58.293755+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33210368,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10656,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13001","reported_seq":13130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.274994+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:17:58.274994+0000","last_peered":"2024-09-05T13:17:58.274994+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:17:58.274994+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33562648,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10812,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2091},{"shards":"2","objects":3703}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12804","reported_seq":12874,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.172109+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:17:58.172109+0000","last_peered":"2024-09-05T13:17:58.172109+0000","last_clean":"2024-09-05T13:17:58.172109+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:17:58.172109+0000","last_undegraded":"2024-09-05T13:17:58.172109+0000","last_fullsized":"2024-09-05T13:17:58.172109+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33677312,"num_objects":8222,"num_object_clones":0,"num_object_copies":16444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12804,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12768","reported_seq":12903,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.705705+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:17:58.705705+0000","last_peered":"2024-09-05T13:17:58.705705+0000","last_clean":"2024-09-05T13:17:58.705705+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:17:58.705705+0000","last_undegraded":"2024-09-05T13:17:58.705705+0000","last_fullsized":"2024-09-05T13:17:58.705705+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9308,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12907","reported_seq":13045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.757669+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:17:58.757669+0000","last_peered":"2024-09-05T13:17:58.757669+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:17:58.757669+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33783808,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9362,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3797},{"shards":"0,2","objects":2124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12625","reported_seq":16594,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.784016+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:17:58.784016+0000","last_peered":"2024-09-05T13:17:58.784016+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:17:58.784016+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":397,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8944,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1914,"num_bytes_recovered":7831552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5479},{"shards":"0,2","objects":2127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12939","reported_seq":13071,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.695373+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:17:58.695373+0000","last_peered":"2024-09-05T13:17:58.695373+0000","last_clean":"2024-09-05T13:17:58.695373+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:17:58.695373+0000","last_undegraded":"2024-09-05T13:17:58.695373+0000","last_fullsized":"2024-09-05T13:17:58.695373+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33443840,"num_objects":8165,"num_object_clones":0,"num_object_copies":16330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9549,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12890","reported_seq":13025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.198785+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:17:58.198785+0000","last_peered":"2024-09-05T13:17:58.198785+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:17:58.198785+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33599488,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10544,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2111},{"shards":"2","objects":3752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12754","reported_seq":12892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.729399+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:17:58.729399+0000","last_peered":"2024-09-05T13:17:58.729399+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:17:58.729399+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9121,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3702},{"shards":"0,2","objects":2159}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12777","reported_seq":21678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:58.175126+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:17:58.175126+0000","last_peered":"2024-09-05T13:17:58.175126+0000","last_clean":"2024-09-05T13:17:58.175126+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:17:58.175126+0000","last_undegraded":"2024-09-05T13:17:58.175126+0000","last_fullsized":"2024-09-05T13:17:58.175126+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33316864,"num_objects":8134,"num_object_clones":0,"num_object_copies":16268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10372,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12584","reported_seq":12707,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.267148+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:17:58.267148+0000","last_peered":"2024-09-05T13:17:58.267148+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:17:58.267148+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32759808,"num_objects":7998,"num_object_clones":0,"num_object_copies":15996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10093,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2097},{"shards":"2","objects":3611}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12775","reported_seq":12916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.658757+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:17:58.658757+0000","last_peered":"2024-09-05T13:17:58.658757+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:17:58.658757+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8926,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3684},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12875","reported_seq":13063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.722979+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:17:58.722979+0000","last_peered":"2024-09-05T13:17:58.722979+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:17:58.722979+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33705984,"num_objects":8229,"num_object_clones":0,"num_object_copies":16458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8774,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12907","reported_seq":13045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.628106+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:17:58.628106+0000","last_peered":"2024-09-05T13:17:58.628106+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:17:58.628106+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33476608,"num_objects":8173,"num_object_clones":0,"num_object_copies":16346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9037,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12973","reported_seq":13113,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:17:58.760624+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:17:58.760624+0000","last_peered":"2024-09-05T13:17:58.760624+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:17:58.760624+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665507,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9641,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":2194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":534503931,"num_objects":130496,"num_object_clones":0,"num_object_copies":260992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45065,"num_objects_misplaced":2127,"num_objects_unfound":0,"num_objects_dirty":130496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160691,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6306,"num_bytes_recovered":25812992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1192124416,"data_stored":1192040399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700960,"kb_used_data":404112,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93670880,"statfs":{"total":96636764160,"available":95918981120,"internally_reserved":0,"allocated":413810688,"data_stored":413551131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":303951533},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847208,"kb_used_data":464320,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93524632,"statfs":{"total":96636764160,"available":95769223168,"internally_reserved":0,"allocated":475463680,"data_stored":475204118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,1,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582680,"kb_used_data":298312,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93789160,"statfs":{"total":96636764160,"available":96040099840,"internally_reserved":0,"allocated":305471488,"data_stored":305142820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473686016,"data_stored":473682403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305045504,"data_stored":304972785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413392896,"data_stored":413385211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:05.435 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:06.024 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:06.051 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-05T13:18:05.638559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525537806,"num_objects":128309,"num_object_clones":0,"num_object_copies":256618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45111,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":128309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162882,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6411,"num_bytes_recovered":26238995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117348,"kb_used_data":1156012,"kb_used_omap":12,"kb_used_meta":961267,"kb_avail":280998172,"statfs":{"total":289910292480,"available":287742128128,"internally_reserved":0,"allocated":1183756288,"data_stored":1182908501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":984337414},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,8,13,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23740416,"num_objects":-5796,"num_object_clones":0,"num_object_copies":-11592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":134,"num_objects_misplaced":-425,"num_objects_unfound":0,"num_objects_dirty":-5796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5796,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":1191936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004978"},"pg_stats":[{"pgid":"2.f","version":"45'12894","reported_seq":12969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.800007+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:03.800007+0000","last_peered":"2024-09-05T13:18:03.800007+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:03.800007+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33087488,"num_objects":8078,"num_object_clones":0,"num_object_copies":16156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12894,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3657},{"shards":"0,2","objects":2012}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12800","reported_seq":12905,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.240141+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:03.240141+0000","last_peered":"2024-09-05T13:18:03.240141+0000","last_clean":"2024-09-05T13:18:03.240141+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:03.240141+0000","last_undegraded":"2024-09-05T13:18:03.240141+0000","last_fullsized":"2024-09-05T13:18:03.240141+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10810,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13138","reported_seq":13267,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.189700+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:03.189700+0000","last_peered":"2024-09-05T13:18:03.189700+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:03.189700+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33001496,"num_objects":8058,"num_object_clones":0,"num_object_copies":16116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10949,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1954},{"shards":"2","objects":3566}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12937","reported_seq":13007,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.190114+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:03.190114+0000","last_peered":"2024-09-05T13:18:03.190114+0000","last_clean":"2024-09-05T13:18:03.190114+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:03.190114+0000","last_undegraded":"2024-09-05T13:18:03.190114+0000","last_fullsized":"2024-09-05T13:18:03.190114+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12937,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12919","reported_seq":13054,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.807958+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:03.807958+0000","last_peered":"2024-09-05T13:18:03.807958+0000","last_clean":"2024-09-05T13:18:03.807958+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:03.807958+0000","last_undegraded":"2024-09-05T13:18:03.807958+0000","last_fullsized":"2024-09-05T13:18:03.807958+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9459,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13049","reported_seq":13188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.756461+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:03.756461+0000","last_peered":"2024-09-05T13:18:03.756461+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:03.756461+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9504,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":1982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12774","reported_seq":16951,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.819250+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:03.819250+0000","last_peered":"2024-09-05T13:18:03.819250+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:03.819250+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":443,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9093,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2018,"num_bytes_recovered":8257536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5434},{"shards":"0,2","objects":1977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13069","reported_seq":13201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.830167+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:03.830167+0000","last_peered":"2024-09-05T13:18:03.830167+0000","last_clean":"2024-09-05T13:18:03.830167+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:03.830167+0000","last_undegraded":"2024-09-05T13:18:03.830167+0000","last_fullsized":"2024-09-05T13:18:03.830167+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32911360,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9679,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13014","reported_seq":13149,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.257947+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:03.257947+0000","last_peered":"2024-09-05T13:18:03.257947+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:03.257947+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10668,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1987},{"shards":"2","objects":3628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12894","reported_seq":13031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.728088+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:03.728088+0000","last_peered":"2024-09-05T13:18:03.728088+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:03.728088+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9261,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3562},{"shards":"0,2","objects":2019}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12891","reported_seq":21792,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.204802+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:03.204802+0000","last_peered":"2024-09-05T13:18:03.204802+0000","last_clean":"2024-09-05T13:18:03.204802+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:03.204802+0000","last_undegraded":"2024-09-05T13:18:03.204802+0000","last_fullsized":"2024-09-05T13:18:03.204802+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32849920,"num_objects":8020,"num_object_clones":0,"num_object_copies":16040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10486,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12702","reported_seq":12825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.109195+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:03.109195+0000","last_peered":"2024-09-05T13:18:03.109195+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:03.109195+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32276480,"num_objects":7880,"num_object_clones":0,"num_object_copies":15760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10211,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1979},{"shards":"2","objects":3493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12909","reported_seq":13049,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.851214+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:03.851214+0000","last_peered":"2024-09-05T13:18:03.851214+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:03.851214+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641024,"num_objects":7969,"num_object_clones":0,"num_object_copies":15938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9060,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3550},{"shards":"0,2","objects":1950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13011","reported_seq":13199,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.808933+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:03.808933+0000","last_peered":"2024-09-05T13:18:03.808933+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:03.808933+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33148928,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8910,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13061","reported_seq":13200,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.793737+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:03.793737+0000","last_peered":"2024-09-05T13:18:03.793737+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:03.793737+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32845824,"num_objects":8019,"num_object_clones":0,"num_object_copies":16038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9191,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3453},{"shards":"0,2","objects":2047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13100","reported_seq":13240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.823770+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:03.823770+0000","last_peered":"2024-09-05T13:18:03.823770+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:03.823770+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33145315,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9768,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3528},{"shards":"0,2","objects":2067}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525537787,"num_objects":128307,"num_object_clones":0,"num_object_copies":256614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45111,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":128307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162880,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6410,"num_bytes_recovered":26238976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180303360,"data_stored":1180219343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680868,"kb_used_data":400652,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93690972,"statfs":{"total":96636764160,"available":95939555328,"internally_reserved":0,"allocated":410267648,"data_stored":410008091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845916,"kb_used_data":457484,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93525924,"statfs":{"total":96636764160,"available":95770546176,"internally_reserved":0,"allocated":468463616,"data_stored":468204054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590564,"kb_used_data":297876,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93781276,"statfs":{"total":96636764160,"available":96032026624,"internally_reserved":0,"allocated":305025024,"data_stored":304696356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466714624,"data_stored":466711011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304574464,"data_stored":304501745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409014272,"data_stored":409006587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:06.053 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:06.619 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:06.619 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:06.638 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-05T13:18:05.638559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525537806,"num_objects":128309,"num_object_clones":0,"num_object_copies":256618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45111,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":128309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162882,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6411,"num_bytes_recovered":26238995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117348,"kb_used_data":1156012,"kb_used_omap":12,"kb_used_meta":961267,"kb_avail":280998172,"statfs":{"total":289910292480,"available":287742128128,"internally_reserved":0,"allocated":1183756288,"data_stored":1182908501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":984337414},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,8,13,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23740416,"num_objects":-5796,"num_object_clones":0,"num_object_copies":-11592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":134,"num_objects_misplaced":-425,"num_objects_unfound":0,"num_objects_dirty":-5796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5796,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":1191936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004978"},"pg_stats":[{"pgid":"2.f","version":"45'12894","reported_seq":12969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.800007+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:03.800007+0000","last_peered":"2024-09-05T13:18:03.800007+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:03.800007+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33087488,"num_objects":8078,"num_object_clones":0,"num_object_copies":16156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12894,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3657},{"shards":"0,2","objects":2012}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12800","reported_seq":12905,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.240141+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:03.240141+0000","last_peered":"2024-09-05T13:18:03.240141+0000","last_clean":"2024-09-05T13:18:03.240141+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:03.240141+0000","last_undegraded":"2024-09-05T13:18:03.240141+0000","last_fullsized":"2024-09-05T13:18:03.240141+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10810,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13138","reported_seq":13267,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.189700+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:03.189700+0000","last_peered":"2024-09-05T13:18:03.189700+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:03.189700+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33001496,"num_objects":8058,"num_object_clones":0,"num_object_copies":16116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10949,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1954},{"shards":"2","objects":3566}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12937","reported_seq":13007,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.190114+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:03.190114+0000","last_peered":"2024-09-05T13:18:03.190114+0000","last_clean":"2024-09-05T13:18:03.190114+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:03.190114+0000","last_undegraded":"2024-09-05T13:18:03.190114+0000","last_fullsized":"2024-09-05T13:18:03.190114+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12937,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12919","reported_seq":13054,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.807958+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:03.807958+0000","last_peered":"2024-09-05T13:18:03.807958+0000","last_clean":"2024-09-05T13:18:03.807958+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:03.807958+0000","last_undegraded":"2024-09-05T13:18:03.807958+0000","last_fullsized":"2024-09-05T13:18:03.807958+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9459,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13049","reported_seq":13188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.756461+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:03.756461+0000","last_peered":"2024-09-05T13:18:03.756461+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:03.756461+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9504,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":1982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12774","reported_seq":16951,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.819250+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:03.819250+0000","last_peered":"2024-09-05T13:18:03.819250+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:03.819250+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":443,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9093,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2018,"num_bytes_recovered":8257536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5434},{"shards":"0,2","objects":1977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13069","reported_seq":13201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.830167+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:03.830167+0000","last_peered":"2024-09-05T13:18:03.830167+0000","last_clean":"2024-09-05T13:18:03.830167+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:03.830167+0000","last_undegraded":"2024-09-05T13:18:03.830167+0000","last_fullsized":"2024-09-05T13:18:03.830167+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32911360,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9679,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13014","reported_seq":13149,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.257947+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:03.257947+0000","last_peered":"2024-09-05T13:18:03.257947+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:03.257947+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10668,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1987},{"shards":"2","objects":3628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12894","reported_seq":13031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.728088+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:03.728088+0000","last_peered":"2024-09-05T13:18:03.728088+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:03.728088+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9261,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3562},{"shards":"0,2","objects":2019}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12891","reported_seq":21792,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.204802+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:03.204802+0000","last_peered":"2024-09-05T13:18:03.204802+0000","last_clean":"2024-09-05T13:18:03.204802+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:03.204802+0000","last_undegraded":"2024-09-05T13:18:03.204802+0000","last_fullsized":"2024-09-05T13:18:03.204802+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32849920,"num_objects":8020,"num_object_clones":0,"num_object_copies":16040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10486,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12702","reported_seq":12825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.109195+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:03.109195+0000","last_peered":"2024-09-05T13:18:03.109195+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:03.109195+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32276480,"num_objects":7880,"num_object_clones":0,"num_object_copies":15760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10211,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1979},{"shards":"2","objects":3493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12909","reported_seq":13049,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.851214+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:03.851214+0000","last_peered":"2024-09-05T13:18:03.851214+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:03.851214+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641024,"num_objects":7969,"num_object_clones":0,"num_object_copies":15938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9060,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3550},{"shards":"0,2","objects":1950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13011","reported_seq":13199,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.808933+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:03.808933+0000","last_peered":"2024-09-05T13:18:03.808933+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:03.808933+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33148928,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8910,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13061","reported_seq":13200,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.793737+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:03.793737+0000","last_peered":"2024-09-05T13:18:03.793737+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:03.793737+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32845824,"num_objects":8019,"num_object_clones":0,"num_object_copies":16038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9191,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3453},{"shards":"0,2","objects":2047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13100","reported_seq":13240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.823770+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:03.823770+0000","last_peered":"2024-09-05T13:18:03.823770+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:03.823770+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33145315,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9768,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3528},{"shards":"0,2","objects":2067}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525537787,"num_objects":128307,"num_object_clones":0,"num_object_copies":256614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45111,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":128307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162880,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6410,"num_bytes_recovered":26238976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180303360,"data_stored":1180219343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680868,"kb_used_data":400652,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93690972,"statfs":{"total":96636764160,"available":95939555328,"internally_reserved":0,"allocated":410267648,"data_stored":410008091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845916,"kb_used_data":457484,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93525924,"statfs":{"total":96636764160,"available":95770546176,"internally_reserved":0,"allocated":468463616,"data_stored":468204054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590564,"kb_used_data":297876,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93781276,"statfs":{"total":96636764160,"available":96032026624,"internally_reserved":0,"allocated":305025024,"data_stored":304696356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466714624,"data_stored":466711011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304574464,"data_stored":304501745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409014272,"data_stored":409006587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:06.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:06.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12894", 'reported_seq': 12969, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.800007+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:03.800007+0000', 'last_peered': '2024-09-05T13:18:03.800007+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:03.800007+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33087488, 'num_objects': 8078, 'num_object_clones': 0, 'num_object_copies': 16156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12894, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3657}, {'shards': '0,2', 'objects': 2012}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:06.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:06.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'13138", 'reported_seq': 13267, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.189700+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:03.189700+0000', 'last_peered': '2024-09-05T13:18:03.189700+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:03.189700+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33001496, 'num_objects': 8058, 'num_object_clones': 0, 'num_object_copies': 16116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8058, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10949, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1954}, {'shards': '2', 'objects': 3566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:06.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:06.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'13049", 'reported_seq': 13188, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.756461+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:03.756461+0000', 'last_peered': '2024-09-05T13:18:03.756461+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:03.756461+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33202176, 'num_objects': 8106, 'num_object_clones': 0, 'num_object_copies': 16212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9504, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3655}, {'shards': '0,2', 'objects': 1982}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:06.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:06.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12774", 'reported_seq': 16951, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.819250+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:03.819250+0000', 'last_peered': '2024-09-05T13:18:03.819250+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:03.819250+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32169984, 'num_objects': 7854, 'num_object_clones': 0, 'num_object_copies': 15708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 443, 'num_objects_misplaced': 1977, 'num_objects_unfound': 0, 'num_objects_dirty': 7854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9093, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2018, 'num_bytes_recovered': 8257536, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5434}, {'shards': '0,2', 'objects': 1977}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'13014", 'reported_seq': 13149, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.257947+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:03.257947+0000', 'last_peered': '2024-09-05T13:18:03.257947+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:03.257947+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33091584, 'num_objects': 8079, 'num_object_clones': 0, 'num_object_copies': 16158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8079, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10668, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1987}, {'shards': '2', 'objects': 3628}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12894", 'reported_seq': 13031, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.728088+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:03.728088+0000', 'last_peered': '2024-09-05T13:18:03.728088+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:03.728088+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32833536, 'num_objects': 8016, 'num_object_clones': 0, 'num_object_copies': 16032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9261, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3562}, {'shards': '0,2', 'objects': 2019}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'12702", 'reported_seq': 12825, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.109195+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:03.109195+0000', 'last_peered': '2024-09-05T13:18:03.109195+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:03.109195+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32276480, 'num_objects': 7880, 'num_object_clones': 0, 'num_object_copies': 15760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7880, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10211, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1979}, {'shards': '2', 'objects': 3493}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12909", 'reported_seq': 13049, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.851214+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:03.851214+0000', 'last_peered': '2024-09-05T13:18:03.851214+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:03.851214+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32641024, 'num_objects': 7969, 'num_object_clones': 0, 'num_object_copies': 15938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7969, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9060, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3550}, {'shards': '0,2', 'objects': 1950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'13011", 'reported_seq': 13199, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.808933+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:03.808933+0000', 'last_peered': '2024-09-05T13:18:03.808933+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:03.808933+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33148928, 'num_objects': 8093, 'num_object_clones': 0, 'num_object_copies': 16186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8910, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3649}, {'shards': '0,2', 'objects': 1989}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:06.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:06.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'13061", 'reported_seq': 13200, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.793737+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:03.793737+0000', 'last_peered': '2024-09-05T13:18:03.793737+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:03.793737+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32845824, 'num_objects': 8019, 'num_object_clones': 0, 'num_object_copies': 16038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9191, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3453}, {'shards': '0,2', 'objects': 2047}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:06.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:06.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'13100", 'reported_seq': 13240, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:03.823770+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:03.823770+0000', 'last_peered': '2024-09-05T13:18:03.823770+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:03.823770+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33145315, 'num_objects': 8093, 'num_object_clones': 0, 'num_object_copies': 16186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9768, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3528}, {'shards': '0,2', 'objects': 2067}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:06.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:07.345 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:07.345 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45111/256618 objects degraded (17.579%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":414,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128309,"data_bytes":525537806,"bytes_used":2168164352,"bytes_avail":287742128128,"bytes_total":289910292480,"degraded_objects":45111,"degraded_total":256618,"degraded_ratio":0.17579047455751351,"misplaced_objects":1977,"misplaced_total":256618,"misplaced_ratio":0.007704058172068989,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":99286,"recovering_keys_per_sec":0,"num_objects_recovered":293,"num_bytes_recovered":1191936,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":482},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:07.365 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:07.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:07.919 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:07.941 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-05T13:18:07.639311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525537806,"num_objects":128309,"num_object_clones":0,"num_object_copies":256618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45111,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":128309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162882,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6411,"num_bytes_recovered":26238995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122388,"kb_used_data":1155480,"kb_used_omap":12,"kb_used_meta":966835,"kb_avail":280993132,"statfs":{"total":289910292480,"available":287736967168,"internally_reserved":0,"allocated":1183211520,"data_stored":1182363733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":990039046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,7,14,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18083840,"num_objects":-4415,"num_object_clones":0,"num_object_copies":-8830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":87,"num_objects_misplaced":-285,"num_objects_unfound":0,"num_objects_dirty":-4415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4415,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":811008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004834"},"pg_stats":[{"pgid":"2.f","version":"45'12894","reported_seq":12969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.800007+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:03.800007+0000","last_peered":"2024-09-05T13:18:03.800007+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:03.800007+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33087488,"num_objects":8078,"num_object_clones":0,"num_object_copies":16156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12894,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3657},{"shards":"0,2","objects":2012}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12800","reported_seq":12905,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.240141+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:03.240141+0000","last_peered":"2024-09-05T13:18:03.240141+0000","last_clean":"2024-09-05T13:18:03.240141+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:03.240141+0000","last_undegraded":"2024-09-05T13:18:03.240141+0000","last_fullsized":"2024-09-05T13:18:03.240141+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32579584,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10810,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13138","reported_seq":13267,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.189700+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:03.189700+0000","last_peered":"2024-09-05T13:18:03.189700+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:03.189700+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33001496,"num_objects":8058,"num_object_clones":0,"num_object_copies":16116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10949,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1954},{"shards":"2","objects":3566}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12937","reported_seq":13007,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.190114+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:03.190114+0000","last_peered":"2024-09-05T13:18:03.190114+0000","last_clean":"2024-09-05T13:18:03.190114+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:03.190114+0000","last_undegraded":"2024-09-05T13:18:03.190114+0000","last_fullsized":"2024-09-05T13:18:03.190114+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12937,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12919","reported_seq":13054,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.807958+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:03.807958+0000","last_peered":"2024-09-05T13:18:03.807958+0000","last_clean":"2024-09-05T13:18:03.807958+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:03.807958+0000","last_undegraded":"2024-09-05T13:18:03.807958+0000","last_fullsized":"2024-09-05T13:18:03.807958+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9459,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13049","reported_seq":13188,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.756461+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:03.756461+0000","last_peered":"2024-09-05T13:18:03.756461+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:03.756461+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9504,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3655},{"shards":"0,2","objects":1982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12774","reported_seq":16951,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.819250+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:03.819250+0000","last_peered":"2024-09-05T13:18:03.819250+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:03.819250+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":443,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9093,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2018,"num_bytes_recovered":8257536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5434},{"shards":"0,2","objects":1977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13069","reported_seq":13201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.830167+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:03.830167+0000","last_peered":"2024-09-05T13:18:03.830167+0000","last_clean":"2024-09-05T13:18:03.830167+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:03.830167+0000","last_undegraded":"2024-09-05T13:18:03.830167+0000","last_fullsized":"2024-09-05T13:18:03.830167+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32911360,"num_objects":8035,"num_object_clones":0,"num_object_copies":16070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9679,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13014","reported_seq":13149,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.257947+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:03.257947+0000","last_peered":"2024-09-05T13:18:03.257947+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:03.257947+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10668,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1987},{"shards":"2","objects":3628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12894","reported_seq":13031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.728088+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:03.728088+0000","last_peered":"2024-09-05T13:18:03.728088+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:03.728088+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":8016,"num_object_clones":0,"num_object_copies":16032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9261,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3562},{"shards":"0,2","objects":2019}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12891","reported_seq":21792,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:03.204802+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:03.204802+0000","last_peered":"2024-09-05T13:18:03.204802+0000","last_clean":"2024-09-05T13:18:03.204802+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:03.204802+0000","last_undegraded":"2024-09-05T13:18:03.204802+0000","last_fullsized":"2024-09-05T13:18:03.204802+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32849920,"num_objects":8020,"num_object_clones":0,"num_object_copies":16040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10486,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12702","reported_seq":12825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.109195+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:03.109195+0000","last_peered":"2024-09-05T13:18:03.109195+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:03.109195+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32276480,"num_objects":7880,"num_object_clones":0,"num_object_copies":15760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10211,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1979},{"shards":"2","objects":3493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12909","reported_seq":13049,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.851214+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:03.851214+0000","last_peered":"2024-09-05T13:18:03.851214+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:03.851214+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32641024,"num_objects":7969,"num_object_clones":0,"num_object_copies":15938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9060,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3550},{"shards":"0,2","objects":1950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13011","reported_seq":13199,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.808933+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:03.808933+0000","last_peered":"2024-09-05T13:18:03.808933+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:03.808933+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33148928,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8910,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13061","reported_seq":13200,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.793737+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:03.793737+0000","last_peered":"2024-09-05T13:18:03.793737+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:03.793737+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32845824,"num_objects":8019,"num_object_clones":0,"num_object_copies":16038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9191,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3453},{"shards":"0,2","objects":2047}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13100","reported_seq":13240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:03.823770+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:03.823770+0000","last_peered":"2024-09-05T13:18:03.823770+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:03.823770+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33145315,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9768,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3528},{"shards":"0,2","objects":2067}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525537787,"num_objects":128307,"num_object_clones":0,"num_object_copies":256614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45111,"num_objects_misplaced":1977,"num_objects_unfound":0,"num_objects_dirty":128307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162880,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6410,"num_bytes_recovered":26238976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1179783168,"data_stored":1179699151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680868,"kb_used_data":400652,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93690972,"statfs":{"total":96636764160,"available":95939555328,"internally_reserved":0,"allocated":410267648,"data_stored":410008091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":286912173},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845916,"kb_used_data":457484,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93525924,"statfs":{"total":96636764160,"available":95770546176,"internally_reserved":0,"allocated":468463616,"data_stored":468204054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595604,"kb_used_data":297344,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93776236,"statfs":{"total":96636764160,"available":96026865664,"internally_reserved":0,"allocated":304480256,"data_stored":304151588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":466714624,"data_stored":466711011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304054272,"data_stored":303981553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409014272,"data_stored":409006587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:10.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:11.497 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:11.524 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-05T13:18:09.640115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516575758,"num_objects":126121,"num_object_clones":0,"num_object_copies":252242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45151,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":126121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165070,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6511,"num_bytes_recovered":26648595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122704,"kb_used_data":1143304,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":280992816,"statfs":{"total":289910292480,"available":287736643584,"internally_reserved":0,"allocated":1170743296,"data_stored":1169895509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1002818566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,5,1,5,16,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27045888,"num_objects":-6603,"num_object_clones":0,"num_object_copies":-13206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":127,"num_objects_misplaced":-425,"num_objects_unfound":0,"num_objects_dirty":-6603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6603,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":1220608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004776"},"pg_stats":[{"pgid":"2.f","version":"45'13024","reported_seq":13099,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.873920+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:08.873920+0000","last_peered":"2024-09-05T13:18:08.873920+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:08.873920+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32555008,"num_objects":7948,"num_object_clones":0,"num_object_copies":15896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13024,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3527},{"shards":"0,2","objects":1882}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12947","reported_seq":13052,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.284995+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:08.284995+0000","last_peered":"2024-09-05T13:18:08.284995+0000","last_clean":"2024-09-05T13:18:08.284995+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:08.284995+0000","last_undegraded":"2024-09-05T13:18:08.284995+0000","last_fullsized":"2024-09-05T13:18:08.284995+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10957,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13279","reported_seq":13408,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.281743+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:08.281743+0000","last_peered":"2024-09-05T13:18:08.281743+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:08.281743+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423960,"num_objects":7917,"num_object_clones":0,"num_object_copies":15834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11090,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1813},{"shards":"2","objects":3425}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13068","reported_seq":13137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.311634+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:08.311634+0000","last_peered":"2024-09-05T13:18:08.311634+0000","last_clean":"2024-09-05T13:18:08.311634+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:08.311634+0000","last_undegraded":"2024-09-05T13:18:08.311634+0000","last_fullsized":"2024-09-05T13:18:08.311634+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13059","reported_seq":13194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.821203+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:08.821203+0000","last_peered":"2024-09-05T13:18:08.821203+0000","last_clean":"2024-09-05T13:18:08.821203+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:08.821203+0000","last_undegraded":"2024-09-05T13:18:08.821203+0000","last_fullsized":"2024-09-05T13:18:08.821203+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13184","reported_seq":13321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.832024+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:08.832024+0000","last_peered":"2024-09-05T13:18:08.832024+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:08.832024+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32649216,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9639,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3520},{"shards":"0,2","objects":1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12911","reported_seq":17288,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.851028+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:08.851028+0000","last_peered":"2024-09-05T13:18:08.851028+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:08.851028+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9230,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":8667136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":1837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13196","reported_seq":13328,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.851402+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:08.851402+0000","last_peered":"2024-09-05T13:18:08.851402+0000","last_clean":"2024-09-05T13:18:08.851402+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:08.851402+0000","last_undegraded":"2024-09-05T13:18:08.851402+0000","last_fullsized":"2024-09-05T13:18:08.851402+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32391168,"num_objects":7908,"num_object_clones":0,"num_object_copies":15816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13127","reported_seq":13262,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.163452+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:08.163452+0000","last_peered":"2024-09-05T13:18:08.163452+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:08.163452+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32628736,"num_objects":7966,"num_object_clones":0,"num_object_copies":15932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1874},{"shards":"2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13046","reported_seq":13178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.806051+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:08.806051+0000","last_peered":"2024-09-05T13:18:08.806051+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:08.806051+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9413,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3410},{"shards":"0,2","objects":1867}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13010","reported_seq":21911,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.239130+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:08.239130+0000","last_peered":"2024-09-05T13:18:08.239130+0000","last_clean":"2024-09-05T13:18:08.239130+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:08.239130+0000","last_undegraded":"2024-09-05T13:18:08.239130+0000","last_fullsized":"2024-09-05T13:18:08.239130+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10605,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12848","reported_seq":12970,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.189551+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:08.189551+0000","last_peered":"2024-09-05T13:18:08.189551+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:08.189551+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10357,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1833},{"shards":"2","objects":3347}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13043","reported_seq":13184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.794035+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:08.794035+0000","last_peered":"2024-09-05T13:18:08.794035+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:08.794035+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32092160,"num_objects":7835,"num_object_clones":0,"num_object_copies":15670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3416},{"shards":"0,2","objects":1816}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13160","reported_seq":13348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.878577+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:08.878577+0000","last_peered":"2024-09-05T13:18:08.878577+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:08.878577+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32538624,"num_objects":7944,"num_object_clones":0,"num_object_copies":15888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9059,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3500},{"shards":"0,2","objects":1840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13207","reported_seq":13343,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.812365+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:08.812365+0000","last_peered":"2024-09-05T13:18:08.812365+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:08.812365+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32247808,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9337,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3307},{"shards":"0,2","objects":1901}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13241","reported_seq":13378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.831618+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:08.831618+0000","last_peered":"2024-09-05T13:18:08.831618+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:08.831618+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567779,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9909,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":1926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516575739,"num_objects":126119,"num_object_clones":0,"num_object_copies":252238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45151,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":126119,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165068,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6510,"num_bytes_recovered":26648576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1168601088,"data_stored":1168517071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680768,"kb_used_data":396400,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93691072,"statfs":{"total":96636764160,"available":95939657728,"internally_reserved":0,"allocated":405913600,"data_stored":405654043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846332,"kb_used_data":449560,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93525508,"statfs":{"total":96636764160,"available":95770120192,"internally_reserved":0,"allocated":460349440,"data_stored":460089878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595604,"kb_used_data":297344,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93776236,"statfs":{"total":96636764160,"available":96026865664,"internally_reserved":0,"allocated":304480256,"data_stored":304151588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459890688,"data_stored":459887075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304054272,"data_stored":303981553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404656128,"data_stored":404648443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:11.526 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:12.072 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:12.074 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:12.102 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-05T13:18:11.640816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516575758,"num_objects":126121,"num_object_clones":0,"num_object_copies":252242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45151,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":126121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165070,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6511,"num_bytes_recovered":26648595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122704,"kb_used_data":1143304,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":280992816,"statfs":{"total":289910292480,"available":287736643584,"internally_reserved":0,"allocated":1170743296,"data_stored":1169895509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1002818566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,5,1,5,16,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17928192,"num_objects":-4377,"num_object_clones":0,"num_object_copies":-8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":-4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4377,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004617"},"pg_stats":[{"pgid":"2.f","version":"45'13024","reported_seq":13099,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.873920+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:08.873920+0000","last_peered":"2024-09-05T13:18:08.873920+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:08.873920+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32555008,"num_objects":7948,"num_object_clones":0,"num_object_copies":15896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13024,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3527},{"shards":"0,2","objects":1882}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12947","reported_seq":13052,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.284995+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:08.284995+0000","last_peered":"2024-09-05T13:18:08.284995+0000","last_clean":"2024-09-05T13:18:08.284995+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:08.284995+0000","last_undegraded":"2024-09-05T13:18:08.284995+0000","last_fullsized":"2024-09-05T13:18:08.284995+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10957,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13279","reported_seq":13408,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.281743+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:08.281743+0000","last_peered":"2024-09-05T13:18:08.281743+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:08.281743+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423960,"num_objects":7917,"num_object_clones":0,"num_object_copies":15834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11090,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1813},{"shards":"2","objects":3425}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13068","reported_seq":13137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.311634+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:08.311634+0000","last_peered":"2024-09-05T13:18:08.311634+0000","last_clean":"2024-09-05T13:18:08.311634+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:08.311634+0000","last_undegraded":"2024-09-05T13:18:08.311634+0000","last_fullsized":"2024-09-05T13:18:08.311634+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13059","reported_seq":13194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.821203+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:08.821203+0000","last_peered":"2024-09-05T13:18:08.821203+0000","last_clean":"2024-09-05T13:18:08.821203+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:08.821203+0000","last_undegraded":"2024-09-05T13:18:08.821203+0000","last_fullsized":"2024-09-05T13:18:08.821203+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13184","reported_seq":13321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.832024+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:08.832024+0000","last_peered":"2024-09-05T13:18:08.832024+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:08.832024+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32649216,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9639,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3520},{"shards":"0,2","objects":1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12911","reported_seq":17288,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.851028+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:08.851028+0000","last_peered":"2024-09-05T13:18:08.851028+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:08.851028+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9230,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":8667136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":1837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13196","reported_seq":13328,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.851402+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:08.851402+0000","last_peered":"2024-09-05T13:18:08.851402+0000","last_clean":"2024-09-05T13:18:08.851402+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:08.851402+0000","last_undegraded":"2024-09-05T13:18:08.851402+0000","last_fullsized":"2024-09-05T13:18:08.851402+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32391168,"num_objects":7908,"num_object_clones":0,"num_object_copies":15816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13127","reported_seq":13262,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.163452+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:08.163452+0000","last_peered":"2024-09-05T13:18:08.163452+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:08.163452+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32628736,"num_objects":7966,"num_object_clones":0,"num_object_copies":15932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1874},{"shards":"2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13046","reported_seq":13178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.806051+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:08.806051+0000","last_peered":"2024-09-05T13:18:08.806051+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:08.806051+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9413,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3410},{"shards":"0,2","objects":1867}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13010","reported_seq":21911,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.239130+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:08.239130+0000","last_peered":"2024-09-05T13:18:08.239130+0000","last_clean":"2024-09-05T13:18:08.239130+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:08.239130+0000","last_undegraded":"2024-09-05T13:18:08.239130+0000","last_fullsized":"2024-09-05T13:18:08.239130+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10605,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12848","reported_seq":12970,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.189551+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:08.189551+0000","last_peered":"2024-09-05T13:18:08.189551+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:08.189551+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10357,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1833},{"shards":"2","objects":3347}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13043","reported_seq":13184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.794035+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:08.794035+0000","last_peered":"2024-09-05T13:18:08.794035+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:08.794035+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32092160,"num_objects":7835,"num_object_clones":0,"num_object_copies":15670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3416},{"shards":"0,2","objects":1816}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13160","reported_seq":13348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.878577+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:08.878577+0000","last_peered":"2024-09-05T13:18:08.878577+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:08.878577+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32538624,"num_objects":7944,"num_object_clones":0,"num_object_copies":15888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9059,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3500},{"shards":"0,2","objects":1840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13207","reported_seq":13343,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.812365+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:08.812365+0000","last_peered":"2024-09-05T13:18:08.812365+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:08.812365+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32247808,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9337,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3307},{"shards":"0,2","objects":1901}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13241","reported_seq":13378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.831618+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:08.831618+0000","last_peered":"2024-09-05T13:18:08.831618+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:08.831618+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567779,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9909,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":1926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516575739,"num_objects":126119,"num_object_clones":0,"num_object_copies":252238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45151,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":126119,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165068,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6510,"num_bytes_recovered":26648576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1168601088,"data_stored":1168517071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680768,"kb_used_data":396400,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93691072,"statfs":{"total":96636764160,"available":95939657728,"internally_reserved":0,"allocated":405913600,"data_stored":405654043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846332,"kb_used_data":449560,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93525508,"statfs":{"total":96636764160,"available":95770120192,"internally_reserved":0,"allocated":460349440,"data_stored":460089878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595604,"kb_used_data":297344,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93776236,"statfs":{"total":96636764160,"available":96026865664,"internally_reserved":0,"allocated":304480256,"data_stored":304151588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459890688,"data_stored":459887075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304054272,"data_stored":303981553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404656128,"data_stored":404648443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:12.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:12.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13024", 'reported_seq': 13099, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.873920+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:08.873920+0000', 'last_peered': '2024-09-05T13:18:08.873920+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:08.873920+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32555008, 'num_objects': 7948, 'num_object_clones': 0, 'num_object_copies': 15896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13024, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3527}, {'shards': '0,2', 'objects': 1882}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:12.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:12.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'13279", 'reported_seq': 13408, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.281743+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:08.281743+0000', 'last_peered': '2024-09-05T13:18:08.281743+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:08.281743+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32423960, 'num_objects': 7917, 'num_object_clones': 0, 'num_object_copies': 15834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7917, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11090, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1813}, {'shards': '2', 'objects': 3425}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:12.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:12.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'13184", 'reported_seq': 13321, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.832024+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:08.832024+0000', 'last_peered': '2024-09-05T13:18:08.832024+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:08.832024+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32649216, 'num_objects': 7971, 'num_object_clones': 0, 'num_object_copies': 15942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9639, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3520}, {'shards': '0,2', 'objects': 1847}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:12.104 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12911", 'reported_seq': 17288, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.851028+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:08.851028+0000', 'last_peered': '2024-09-05T13:18:08.851028+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:08.851028+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31608832, 'num_objects': 7717, 'num_object_clones': 0, 'num_object_copies': 15434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 483, 'num_objects_misplaced': 1837, 'num_objects_unfound': 0, 'num_objects_dirty': 7717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9230, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2118, 'num_bytes_recovered': 8667136, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5397}, {'shards': '0,2', 'objects': 1837}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'13127", 'reported_seq': 13262, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.163452+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:08.163452+0000', 'last_peered': '2024-09-05T13:18:08.163452+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:08.163452+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32628736, 'num_objects': 7966, 'num_object_clones': 0, 'num_object_copies': 15932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10781, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1874}, {'shards': '2', 'objects': 3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'13046", 'reported_seq': 13178, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.806051+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:08.806051+0000', 'last_peered': '2024-09-05T13:18:08.806051+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:08.806051+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32210944, 'num_objects': 7864, 'num_object_clones': 0, 'num_object_copies': 15728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9413, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3410}, {'shards': '0,2', 'objects': 1867}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'12848", 'reported_seq': 12970, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.189551+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:08.189551+0000', 'last_peered': '2024-09-05T13:18:08.189551+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:08.189551+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31678464, 'num_objects': 7734, 'num_object_clones': 0, 'num_object_copies': 15468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10357, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1833}, {'shards': '2', 'objects': 3347}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'13043", 'reported_seq': 13184, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.794035+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:08.794035+0000', 'last_peered': '2024-09-05T13:18:08.794035+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:08.794035+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32092160, 'num_objects': 7835, 'num_object_clones': 0, 'num_object_copies': 15670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9194, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3416}, {'shards': '0,2', 'objects': 1816}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:12.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'13160", 'reported_seq': 13348, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.878577+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:08.878577+0000', 'last_peered': '2024-09-05T13:18:08.878577+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:08.878577+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32538624, 'num_objects': 7944, 'num_object_clones': 0, 'num_object_copies': 15888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7944, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9059, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3500}, {'shards': '0,2', 'objects': 1840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:12.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:12.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'13207", 'reported_seq': 13343, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.812365+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:08.812365+0000', 'last_peered': '2024-09-05T13:18:08.812365+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:08.812365+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32247808, 'num_objects': 7873, 'num_object_clones': 0, 'num_object_copies': 15746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9337, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3307}, {'shards': '0,2', 'objects': 1901}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:12.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:12.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'13241", 'reported_seq': 13378, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:08.831618+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:08.831618+0000', 'last_peered': '2024-09-05T13:18:08.831618+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:08.831618+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32567779, 'num_objects': 7952, 'num_object_clones': 0, 'num_object_copies': 15904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9909, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3387}, {'shards': '0,2', 'objects': 1926}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:12.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:12.855 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:12.855 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45151/252242 objects degraded (17.900%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":419,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":126121,"data_bytes":516575758,"bytes_used":2173648896,"bytes_avail":287736643584,"bytes_total":289910292480,"degraded_objects":45151,"degraded_total":252242,"degraded_ratio":0.17899873930590465,"misplaced_objects":1837,"misplaced_total":252242,"misplaced_ratio":0.0072826888464252578,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":69605,"recovering_keys_per_sec":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":364},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:12.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:12.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:13.411 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:13.411 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:13.444 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-05T13:18:11.640816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516575758,"num_objects":126121,"num_object_clones":0,"num_object_copies":252242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45151,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":126121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165070,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6511,"num_bytes_recovered":26648595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122704,"kb_used_data":1143304,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":280992816,"statfs":{"total":289910292480,"available":287736643584,"internally_reserved":0,"allocated":1170743296,"data_stored":1169895509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1002818566},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,5,1,5,16,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17928192,"num_objects":-4377,"num_object_clones":0,"num_object_copies":-8754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":-4377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4377,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004617"},"pg_stats":[{"pgid":"2.f","version":"45'13024","reported_seq":13099,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.873920+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:08.873920+0000","last_peered":"2024-09-05T13:18:08.873920+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:08.873920+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32555008,"num_objects":7948,"num_object_clones":0,"num_object_copies":15896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13024,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3527},{"shards":"0,2","objects":1882}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12947","reported_seq":13052,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.284995+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:08.284995+0000","last_peered":"2024-09-05T13:18:08.284995+0000","last_clean":"2024-09-05T13:18:08.284995+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:08.284995+0000","last_undegraded":"2024-09-05T13:18:08.284995+0000","last_fullsized":"2024-09-05T13:18:08.284995+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10957,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13279","reported_seq":13408,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.281743+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:08.281743+0000","last_peered":"2024-09-05T13:18:08.281743+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:08.281743+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423960,"num_objects":7917,"num_object_clones":0,"num_object_copies":15834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11090,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1813},{"shards":"2","objects":3425}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13068","reported_seq":13137,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.311634+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:08.311634+0000","last_peered":"2024-09-05T13:18:08.311634+0000","last_clean":"2024-09-05T13:18:08.311634+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:08.311634+0000","last_undegraded":"2024-09-05T13:18:08.311634+0000","last_fullsized":"2024-09-05T13:18:08.311634+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13059","reported_seq":13194,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.821203+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:08.821203+0000","last_peered":"2024-09-05T13:18:08.821203+0000","last_clean":"2024-09-05T13:18:08.821203+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:08.821203+0000","last_undegraded":"2024-09-05T13:18:08.821203+0000","last_fullsized":"2024-09-05T13:18:08.821203+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13184","reported_seq":13321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.832024+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:08.832024+0000","last_peered":"2024-09-05T13:18:08.832024+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:08.832024+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32649216,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9639,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3520},{"shards":"0,2","objects":1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12911","reported_seq":17288,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.851028+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:08.851028+0000","last_peered":"2024-09-05T13:18:08.851028+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:08.851028+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":483,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9230,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2118,"num_bytes_recovered":8667136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":1837}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13196","reported_seq":13328,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.851402+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:08.851402+0000","last_peered":"2024-09-05T13:18:08.851402+0000","last_clean":"2024-09-05T13:18:08.851402+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:08.851402+0000","last_undegraded":"2024-09-05T13:18:08.851402+0000","last_fullsized":"2024-09-05T13:18:08.851402+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32391168,"num_objects":7908,"num_object_clones":0,"num_object_copies":15816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13127","reported_seq":13262,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.163452+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:08.163452+0000","last_peered":"2024-09-05T13:18:08.163452+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:08.163452+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32628736,"num_objects":7966,"num_object_clones":0,"num_object_copies":15932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1874},{"shards":"2","objects":3515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13046","reported_seq":13178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.806051+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:08.806051+0000","last_peered":"2024-09-05T13:18:08.806051+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:08.806051+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9413,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3410},{"shards":"0,2","objects":1867}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13010","reported_seq":21911,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:08.239130+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:08.239130+0000","last_peered":"2024-09-05T13:18:08.239130+0000","last_clean":"2024-09-05T13:18:08.239130+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:08.239130+0000","last_undegraded":"2024-09-05T13:18:08.239130+0000","last_fullsized":"2024-09-05T13:18:08.239130+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10605,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12848","reported_seq":12970,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.189551+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:08.189551+0000","last_peered":"2024-09-05T13:18:08.189551+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:08.189551+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10357,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1833},{"shards":"2","objects":3347}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13043","reported_seq":13184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.794035+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:08.794035+0000","last_peered":"2024-09-05T13:18:08.794035+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:08.794035+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32092160,"num_objects":7835,"num_object_clones":0,"num_object_copies":15670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9194,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3416},{"shards":"0,2","objects":1816}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13160","reported_seq":13348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.878577+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:08.878577+0000","last_peered":"2024-09-05T13:18:08.878577+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:08.878577+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32538624,"num_objects":7944,"num_object_clones":0,"num_object_copies":15888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9059,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3500},{"shards":"0,2","objects":1840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13207","reported_seq":13343,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.812365+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:08.812365+0000","last_peered":"2024-09-05T13:18:08.812365+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:08.812365+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32247808,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9337,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3307},{"shards":"0,2","objects":1901}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13241","reported_seq":13378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:08.831618+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:08.831618+0000","last_peered":"2024-09-05T13:18:08.831618+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:08.831618+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567779,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9909,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3387},{"shards":"0,2","objects":1926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516575739,"num_objects":126119,"num_object_clones":0,"num_object_copies":252238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45151,"num_objects_misplaced":1837,"num_objects_unfound":0,"num_objects_dirty":126119,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165068,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6510,"num_bytes_recovered":26648576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1168601088,"data_stored":1168517071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680768,"kb_used_data":396400,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93691072,"statfs":{"total":96636764160,"available":95939657728,"internally_reserved":0,"allocated":405913600,"data_stored":405654043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":291172013},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":846332,"kb_used_data":449560,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93525508,"statfs":{"total":96636764160,"available":95770120192,"internally_reserved":0,"allocated":460349440,"data_stored":460089878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595604,"kb_used_data":297344,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93776236,"statfs":{"total":96636764160,"available":96026865664,"internally_reserved":0,"allocated":304480256,"data_stored":304151588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":459890688,"data_stored":459887075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304054272,"data_stored":303981553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404656128,"data_stored":404648443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:16.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:17.032 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:17.033 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:17.062 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-05T13:18:15.643883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507519502,"num_objects":123910,"num_object_clones":0,"num_object_copies":247820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45183,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":123910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167281,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":27090963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160964,"ondisk_log_size":160964,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127752,"kb_used_data":1131736,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":280987768,"statfs":{"total":289910292480,"available":287731474432,"internally_reserved":0,"allocated":1158897664,"data_stored":1158049877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1019857926},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,5,1,7,3,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23789568,"num_objects":-5808,"num_object_clones":0,"num_object_copies":-11616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":-430,"num_objects_unfound":0,"num_objects_dirty":-5808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5808,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006097"},"pg_stats":[{"pgid":"2.f","version":"45'13166","reported_seq":13241,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.874127+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:13.874127+0000","last_peered":"2024-09-05T13:18:13.874127+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:13.874127+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31973376,"num_objects":7806,"num_object_clones":0,"num_object_copies":15612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3385},{"shards":"0,2","objects":1740}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13096","reported_seq":13201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.308171+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:13.308171+0000","last_peered":"2024-09-05T13:18:13.308171+0000","last_clean":"2024-09-05T13:18:13.308171+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:13.308171+0000","last_undegraded":"2024-09-05T13:18:13.308171+0000","last_fullsized":"2024-09-05T13:18:13.308171+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11106,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13391","reported_seq":13520,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.303968+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:13.303968+0000","last_peered":"2024-09-05T13:18:13.303968+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:13.303968+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31965208,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11202,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1701},{"shards":"2","objects":3313}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13194","reported_seq":13264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.286117+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:13.286117+0000","last_peered":"2024-09-05T13:18:13.286117+0000","last_clean":"2024-09-05T13:18:13.286117+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:13.286117+0000","last_undegraded":"2024-09-05T13:18:13.286117+0000","last_fullsized":"2024-09-05T13:18:13.286117+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32079872,"num_objects":7832,"num_object_clones":0,"num_object_copies":15664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13194,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13206","reported_seq":13341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.917001+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:13.917001+0000","last_peered":"2024-09-05T13:18:13.917001+0000","last_clean":"2024-09-05T13:18:13.917001+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:13.917001+0000","last_undegraded":"2024-09-05T13:18:13.917001+0000","last_fullsized":"2024-09-05T13:18:13.917001+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9746,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13314","reported_seq":13452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.886742+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:13.886742+0000","last_peered":"2024-09-05T13:18:13.886742+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:13.886742+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32116736,"num_objects":7841,"num_object_clones":0,"num_object_copies":15682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9769,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3390},{"shards":"0,2","objects":1717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13054","reported_seq":17647,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.908034+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:13.908034+0000","last_peered":"2024-09-05T13:18:13.908034+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:13.908034+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31023104,"num_objects":7574,"num_object_clones":0,"num_object_copies":15148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":515,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":7574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2226,"num_bytes_recovered":9109504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5362},{"shards":"0,2","objects":1697}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13330","reported_seq":13462,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.622647+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:13.622647+0000","last_peered":"2024-09-05T13:18:13.622647+0000","last_clean":"2024-09-05T13:18:13.622647+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:13.622647+0000","last_undegraded":"2024-09-05T13:18:13.622647+0000","last_fullsized":"2024-09-05T13:18:13.622647+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31842304,"num_objects":7774,"num_object_clones":0,"num_object_copies":15548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9940,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13269","reported_seq":13404,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.185523+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:13.185523+0000","last_peered":"2024-09-05T13:18:13.185523+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:13.185523+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10923,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1732},{"shards":"2","objects":3373}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13201","reported_seq":13337,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.885150+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:13.885150+0000","last_peered":"2024-09-05T13:18:13.885150+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:13.885150+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9568,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3255},{"shards":"0,2","objects":1712}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13157","reported_seq":22058,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.306023+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:13.306023+0000","last_peered":"2024-09-05T13:18:13.306023+0000","last_clean":"2024-09-05T13:18:13.306023+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:13.306023+0000","last_undegraded":"2024-09-05T13:18:13.306023+0000","last_fullsized":"2024-09-05T13:18:13.306023+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31760384,"num_objects":7754,"num_object_clones":0,"num_object_copies":15508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10752,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12995","reported_seq":13118,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.306277+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:13.306277+0000","last_peered":"2024-09-05T13:18:13.306277+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:13.306277+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31076352,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10504,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1686},{"shards":"2","objects":3200}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13189","reported_seq":13330,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.890793+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:13.890793+0000","last_peered":"2024-09-05T13:18:13.890793+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:13.890793+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9340,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3270},{"shards":"0,2","objects":1670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13289","reported_seq":13477,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.863204+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:13.863204+0000","last_peered":"2024-09-05T13:18:13.863204+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:13.863204+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32010240,"num_objects":7815,"num_object_clones":0,"num_object_copies":15630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9188,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13340","reported_seq":13478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.871109+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:13.871109+0000","last_peered":"2024-09-05T13:18:13.871109+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:13.871109+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31703040,"num_objects":7740,"num_object_clones":0,"num_object_copies":15480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9470,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3174},{"shards":"0,2","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13370","reported_seq":13510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.889968+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:13.889968+0000","last_peered":"2024-09-05T13:18:13.889968+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:13.889968+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32039395,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10038,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3258},{"shards":"0,2","objects":1797}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507519483,"num_objects":123908,"num_object_clones":0,"num_object_copies":247816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45183,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":123908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167279,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6618,"num_bytes_recovered":27090944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1156673536,"data_stored":1156589519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680396,"kb_used_data":391868,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93691444,"statfs":{"total":96636764160,"available":95940038656,"internally_reserved":0,"allocated":401272832,"data_stored":401013275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843992,"kb_used_data":443080,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93527848,"statfs":{"total":96636764160,"available":95772516352,"internally_reserved":0,"allocated":453713920,"data_stored":453454358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,1,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603364,"kb_used_data":296788,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93768476,"statfs":{"total":96636764160,"available":96018919424,"internally_reserved":0,"allocated":303910912,"data_stored":303582244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453017600,"data_stored":453013987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303480832,"data_stored":303408113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400175104,"data_stored":400167419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:17.063 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:17.618 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:17.619 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:17.644 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-05T13:18:15.643883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507519502,"num_objects":123910,"num_object_clones":0,"num_object_copies":247820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45183,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":123910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167281,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":27090963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160964,"ondisk_log_size":160964,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127752,"kb_used_data":1131736,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":280987768,"statfs":{"total":289910292480,"available":287731474432,"internally_reserved":0,"allocated":1158897664,"data_stored":1158049877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1019857926},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,5,1,7,3,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23789568,"num_objects":-5808,"num_object_clones":0,"num_object_copies":-11616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":118,"num_objects_misplaced":-430,"num_objects_unfound":0,"num_objects_dirty":-5808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5808,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006097"},"pg_stats":[{"pgid":"2.f","version":"45'13166","reported_seq":13241,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.874127+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:13.874127+0000","last_peered":"2024-09-05T13:18:13.874127+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:13.874127+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31973376,"num_objects":7806,"num_object_clones":0,"num_object_copies":15612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3385},{"shards":"0,2","objects":1740}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13096","reported_seq":13201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.308171+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:13.308171+0000","last_peered":"2024-09-05T13:18:13.308171+0000","last_clean":"2024-09-05T13:18:13.308171+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:13.308171+0000","last_undegraded":"2024-09-05T13:18:13.308171+0000","last_fullsized":"2024-09-05T13:18:13.308171+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11106,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13391","reported_seq":13520,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.303968+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:13.303968+0000","last_peered":"2024-09-05T13:18:13.303968+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:13.303968+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31965208,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11202,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1701},{"shards":"2","objects":3313}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13194","reported_seq":13264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.286117+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:13.286117+0000","last_peered":"2024-09-05T13:18:13.286117+0000","last_clean":"2024-09-05T13:18:13.286117+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:13.286117+0000","last_undegraded":"2024-09-05T13:18:13.286117+0000","last_fullsized":"2024-09-05T13:18:13.286117+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32079872,"num_objects":7832,"num_object_clones":0,"num_object_copies":15664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13194,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13206","reported_seq":13341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.917001+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:13.917001+0000","last_peered":"2024-09-05T13:18:13.917001+0000","last_clean":"2024-09-05T13:18:13.917001+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:13.917001+0000","last_undegraded":"2024-09-05T13:18:13.917001+0000","last_fullsized":"2024-09-05T13:18:13.917001+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9746,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13314","reported_seq":13452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.886742+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:13.886742+0000","last_peered":"2024-09-05T13:18:13.886742+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:13.886742+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32116736,"num_objects":7841,"num_object_clones":0,"num_object_copies":15682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9769,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3390},{"shards":"0,2","objects":1717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13054","reported_seq":17647,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.908034+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:13.908034+0000","last_peered":"2024-09-05T13:18:13.908034+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:13.908034+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31023104,"num_objects":7574,"num_object_clones":0,"num_object_copies":15148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":515,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":7574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2226,"num_bytes_recovered":9109504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5362},{"shards":"0,2","objects":1697}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13330","reported_seq":13462,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.622647+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:13.622647+0000","last_peered":"2024-09-05T13:18:13.622647+0000","last_clean":"2024-09-05T13:18:13.622647+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:13.622647+0000","last_undegraded":"2024-09-05T13:18:13.622647+0000","last_fullsized":"2024-09-05T13:18:13.622647+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31842304,"num_objects":7774,"num_object_clones":0,"num_object_copies":15548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9940,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13269","reported_seq":13404,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.185523+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:13.185523+0000","last_peered":"2024-09-05T13:18:13.185523+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:13.185523+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10923,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1732},{"shards":"2","objects":3373}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13201","reported_seq":13337,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.885150+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:13.885150+0000","last_peered":"2024-09-05T13:18:13.885150+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:13.885150+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9568,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3255},{"shards":"0,2","objects":1712}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13157","reported_seq":22058,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.306023+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:13.306023+0000","last_peered":"2024-09-05T13:18:13.306023+0000","last_clean":"2024-09-05T13:18:13.306023+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:13.306023+0000","last_undegraded":"2024-09-05T13:18:13.306023+0000","last_fullsized":"2024-09-05T13:18:13.306023+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31760384,"num_objects":7754,"num_object_clones":0,"num_object_copies":15508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10752,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12995","reported_seq":13118,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.306277+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:13.306277+0000","last_peered":"2024-09-05T13:18:13.306277+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:13.306277+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31076352,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10504,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1686},{"shards":"2","objects":3200}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13189","reported_seq":13330,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.890793+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:13.890793+0000","last_peered":"2024-09-05T13:18:13.890793+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:13.890793+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9340,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3270},{"shards":"0,2","objects":1670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13289","reported_seq":13477,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.863204+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:13.863204+0000","last_peered":"2024-09-05T13:18:13.863204+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:13.863204+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32010240,"num_objects":7815,"num_object_clones":0,"num_object_copies":15630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9188,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13340","reported_seq":13478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.871109+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:13.871109+0000","last_peered":"2024-09-05T13:18:13.871109+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:13.871109+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31703040,"num_objects":7740,"num_object_clones":0,"num_object_copies":15480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9470,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3174},{"shards":"0,2","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13370","reported_seq":13510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.889968+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:13.889968+0000","last_peered":"2024-09-05T13:18:13.889968+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:13.889968+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32039395,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10038,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3258},{"shards":"0,2","objects":1797}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507519483,"num_objects":123908,"num_object_clones":0,"num_object_copies":247816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45183,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":123908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167279,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6618,"num_bytes_recovered":27090944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1156673536,"data_stored":1156589519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680396,"kb_used_data":391868,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93691444,"statfs":{"total":96636764160,"available":95940038656,"internally_reserved":0,"allocated":401272832,"data_stored":401013275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843992,"kb_used_data":443080,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93527848,"statfs":{"total":96636764160,"available":95772516352,"internally_reserved":0,"allocated":453713920,"data_stored":453454358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,1,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603364,"kb_used_data":296788,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93768476,"statfs":{"total":96636764160,"available":96018919424,"internally_reserved":0,"allocated":303910912,"data_stored":303582244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453017600,"data_stored":453013987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303480832,"data_stored":303408113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400175104,"data_stored":400167419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:17.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:17.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13166", 'reported_seq': 13241, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.874127+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:13.874127+0000', 'last_peered': '2024-09-05T13:18:13.874127+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:13.874127+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31973376, 'num_objects': 7806, 'num_object_clones': 0, 'num_object_copies': 15612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7806, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13166, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3385}, {'shards': '0,2', 'objects': 1740}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:17.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:17.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'13391", 'reported_seq': 13520, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.303968+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:13.303968+0000', 'last_peered': '2024-09-05T13:18:13.303968+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:13.303968+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31965208, 'num_objects': 7805, 'num_object_clones': 0, 'num_object_copies': 15610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7805, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11202, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1701}, {'shards': '2', 'objects': 3313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'13314", 'reported_seq': 13452, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.886742+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:13.886742+0000', 'last_peered': '2024-09-05T13:18:13.886742+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:13.886742+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32116736, 'num_objects': 7841, 'num_object_clones': 0, 'num_object_copies': 15682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9769, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3390}, {'shards': '0,2', 'objects': 1717}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'13054", 'reported_seq': 17647, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.908034+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:13.908034+0000', 'last_peered': '2024-09-05T13:18:13.908034+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:13.908034+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31023104, 'num_objects': 7574, 'num_object_clones': 0, 'num_object_copies': 15148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 515, 'num_objects_misplaced': 1697, 'num_objects_unfound': 0, 'num_objects_dirty': 7574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9373, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2226, 'num_bytes_recovered': 9109504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5362}, {'shards': '0,2', 'objects': 1697}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'13269", 'reported_seq': 13404, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.185523+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:13.185523+0000', 'last_peered': '2024-09-05T13:18:13.185523+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:13.185523+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32047104, 'num_objects': 7824, 'num_object_clones': 0, 'num_object_copies': 15648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10923, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1732}, {'shards': '2', 'objects': 3373}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'13201", 'reported_seq': 13337, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.885150+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:13.885150+0000', 'last_peered': '2024-09-05T13:18:13.885150+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:13.885150+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31576064, 'num_objects': 7709, 'num_object_clones': 0, 'num_object_copies': 15418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9568, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3255}, {'shards': '0,2', 'objects': 1712}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:17.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'12995", 'reported_seq': 13118, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.306277+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:13.306277+0000', 'last_peered': '2024-09-05T13:18:13.306277+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:13.306277+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31076352, 'num_objects': 7587, 'num_object_clones': 0, 'num_object_copies': 15174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10504, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1686}, {'shards': '2', 'objects': 3200}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:17.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:17.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'13189", 'reported_seq': 13330, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.890793+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:13.890793+0000', 'last_peered': '2024-09-05T13:18:13.890793+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:13.890793+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31494144, 'num_objects': 7689, 'num_object_clones': 0, 'num_object_copies': 15378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7689, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9340, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3270}, {'shards': '0,2', 'objects': 1670}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:17.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:17.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'13289", 'reported_seq': 13477, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.863204+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:13.863204+0000', 'last_peered': '2024-09-05T13:18:13.863204+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:13.863204+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32010240, 'num_objects': 7815, 'num_object_clones': 0, 'num_object_copies': 15630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7815, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9188, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3371}, {'shards': '0,2', 'objects': 1711}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:17.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:17.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'13340", 'reported_seq': 13478, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.871109+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:13.871109+0000', 'last_peered': '2024-09-05T13:18:13.871109+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:13.871109+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31703040, 'num_objects': 7740, 'num_object_clones': 0, 'num_object_copies': 15480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7740, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9470, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3174}, {'shards': '0,2', 'objects': 1768}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:17.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:17.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'13370", 'reported_seq': 13510, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:13.889968+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:13.889968+0000', 'last_peered': '2024-09-05T13:18:13.889968+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:13.889968+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32039395, 'num_objects': 7823, 'num_object_clones': 0, 'num_object_copies': 15646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10038, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3258}, {'shards': '0,2', 'objects': 1797}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:17.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:18.280 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:18.280 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45183/247820 objects degraded (18.232%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":425,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123910,"data_bytes":507519502,"bytes_used":2182545408,"bytes_avail":287727747072,"bytes_total":289910292480,"degraded_objects":45183,"degraded_total":247820,"degraded_ratio":0.18232184650149302,"misplaced_objects":1697,"misplaced_total":247820,"misplaced_ratio":0.0068477120490678719,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":70961,"recovering_keys_per_sec":0,"num_objects_recovered":208,"num_bytes_recovered":851968,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":366},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:18.300 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:18.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:18.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:18.841 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:18.880 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-05T13:18:17.644548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":507519502,"num_objects":123910,"num_object_clones":0,"num_object_copies":247820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45183,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":123910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167281,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6619,"num_bytes_recovered":27090963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160964,"ondisk_log_size":160964,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2131392,"kb_used_data":1131216,"kb_used_omap":12,"kb_used_meta":1000115,"kb_avail":280984128,"statfs":{"total":289910292480,"available":287727747072,"internally_reserved":0,"allocated":1158365184,"data_stored":1157517397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1024117766},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,5,0,8,5,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18018304,"num_objects":-4399,"num_object_clones":0,"num_object_copies":-8798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":-280,"num_objects_unfound":0,"num_objects_dirty":-4399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4399,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005989"},"pg_stats":[{"pgid":"2.f","version":"45'13166","reported_seq":13241,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.874127+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:13.874127+0000","last_peered":"2024-09-05T13:18:13.874127+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:13.874127+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31973376,"num_objects":7806,"num_object_clones":0,"num_object_copies":15612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3385},{"shards":"0,2","objects":1740}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13096","reported_seq":13201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.308171+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:13.308171+0000","last_peered":"2024-09-05T13:18:13.308171+0000","last_clean":"2024-09-05T13:18:13.308171+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:13.308171+0000","last_undegraded":"2024-09-05T13:18:13.308171+0000","last_fullsized":"2024-09-05T13:18:13.308171+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31367168,"num_objects":7658,"num_object_clones":0,"num_object_copies":15316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11106,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13391","reported_seq":13520,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.303968+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:13.303968+0000","last_peered":"2024-09-05T13:18:13.303968+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:13.303968+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31965208,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11202,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1701},{"shards":"2","objects":3313}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13194","reported_seq":13264,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.286117+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:13.286117+0000","last_peered":"2024-09-05T13:18:13.286117+0000","last_clean":"2024-09-05T13:18:13.286117+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:13.286117+0000","last_undegraded":"2024-09-05T13:18:13.286117+0000","last_fullsized":"2024-09-05T13:18:13.286117+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32079872,"num_objects":7832,"num_object_clones":0,"num_object_copies":15664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13194,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13206","reported_seq":13341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.917001+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:13.917001+0000","last_peered":"2024-09-05T13:18:13.917001+0000","last_clean":"2024-09-05T13:18:13.917001+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:13.917001+0000","last_undegraded":"2024-09-05T13:18:13.917001+0000","last_fullsized":"2024-09-05T13:18:13.917001+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9746,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13314","reported_seq":13452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.886742+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:13.886742+0000","last_peered":"2024-09-05T13:18:13.886742+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:13.886742+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32116736,"num_objects":7841,"num_object_clones":0,"num_object_copies":15682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9769,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3390},{"shards":"0,2","objects":1717}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13054","reported_seq":17647,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.908034+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:13.908034+0000","last_peered":"2024-09-05T13:18:13.908034+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:13.908034+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31023104,"num_objects":7574,"num_object_clones":0,"num_object_copies":15148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":515,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":7574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9373,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2226,"num_bytes_recovered":9109504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5362},{"shards":"0,2","objects":1697}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13330","reported_seq":13462,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.622647+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:13.622647+0000","last_peered":"2024-09-05T13:18:13.622647+0000","last_clean":"2024-09-05T13:18:13.622647+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:13.622647+0000","last_undegraded":"2024-09-05T13:18:13.622647+0000","last_fullsized":"2024-09-05T13:18:13.622647+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31842304,"num_objects":7774,"num_object_clones":0,"num_object_copies":15548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9940,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13269","reported_seq":13404,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.185523+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:13.185523+0000","last_peered":"2024-09-05T13:18:13.185523+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:13.185523+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10923,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1732},{"shards":"2","objects":3373}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13201","reported_seq":13337,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.885150+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:13.885150+0000","last_peered":"2024-09-05T13:18:13.885150+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:13.885150+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9568,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3255},{"shards":"0,2","objects":1712}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.202093+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:09.202093+0000","last_peered":"2024-09-05T13:16:09.202093+0000","last_clean":"2024-09-05T13:16:09.202093+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:09.202093+0000","last_undegraded":"2024-09-05T13:16:09.202093+0000","last_fullsized":"2024-09-05T13:16:09.202093+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13157","reported_seq":22058,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:13.306023+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:13.306023+0000","last_peered":"2024-09-05T13:18:13.306023+0000","last_clean":"2024-09-05T13:18:13.306023+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:13.306023+0000","last_undegraded":"2024-09-05T13:18:13.306023+0000","last_fullsized":"2024-09-05T13:18:13.306023+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31760384,"num_objects":7754,"num_object_clones":0,"num_object_copies":15508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10752,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12995","reported_seq":13118,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.306277+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:13.306277+0000","last_peered":"2024-09-05T13:18:13.306277+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:13.306277+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31076352,"num_objects":7587,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10504,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1686},{"shards":"2","objects":3200}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13189","reported_seq":13330,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.890793+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:13.890793+0000","last_peered":"2024-09-05T13:18:13.890793+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:13.890793+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9340,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3270},{"shards":"0,2","objects":1670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13289","reported_seq":13477,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.863204+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:13.863204+0000","last_peered":"2024-09-05T13:18:13.863204+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:13.863204+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32010240,"num_objects":7815,"num_object_clones":0,"num_object_copies":15630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9188,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3371},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13340","reported_seq":13478,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.871109+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:13.871109+0000","last_peered":"2024-09-05T13:18:13.871109+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:13.871109+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31703040,"num_objects":7740,"num_object_clones":0,"num_object_copies":15480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9470,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3174},{"shards":"0,2","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13370","reported_seq":13510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:13.889968+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:13.889968+0000","last_peered":"2024-09-05T13:18:13.889968+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:13.889968+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32039395,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10038,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3258},{"shards":"0,2","objects":1797}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":507519483,"num_objects":123908,"num_object_clones":0,"num_object_copies":247816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45183,"num_objects_misplaced":1697,"num_objects_unfound":0,"num_objects_dirty":123908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167279,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6618,"num_bytes_recovered":27090944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1156149248,"data_stored":1156065231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160961,"ondisk_log_size":160961,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680396,"kb_used_data":391868,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93691444,"statfs":{"total":96636764160,"available":95940038656,"internally_reserved":0,"allocated":401272832,"data_stored":401013275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":295431853},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843992,"kb_used_data":443080,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93527848,"statfs":{"total":96636764160,"available":95772516352,"internally_reserved":0,"allocated":453713920,"data_stored":453454358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,1,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607004,"kb_used_data":296268,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93764836,"statfs":{"total":96636764160,"available":96015192064,"internally_reserved":0,"allocated":303378432,"data_stored":303049764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453017600,"data_stored":453013987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":302956544,"data_stored":302883825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400175104,"data_stored":400167419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:21.882 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:22.443 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:22.473 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-05T13:18:21.645952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":498655758,"num_objects":121746,"num_object_clones":0,"num_object_copies":243492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45216,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":121746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169445,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6727,"num_bytes_recovered":27533331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2130160,"kb_used_data":1120248,"kb_used_omap":12,"kb_used_meta":1009843,"kb_avail":280985360,"statfs":{"total":289910292480,"available":287729008640,"internally_reserved":0,"allocated":1147133952,"data_stored":1146286165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1034079238},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,6,2,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17920000,"num_objects":-4375,"num_object_clones":0,"num_object_copies":-8750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":65,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":-4375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4375,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005837"},"pg_stats":[{"pgid":"2.f","version":"45'13295","reported_seq":13370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.001888+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:19.001888+0000","last_peered":"2024-09-05T13:18:19.001888+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:19.001888+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13295,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3256},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13243","reported_seq":13348,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.260289+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:18.260289+0000","last_peered":"2024-09-05T13:18:18.260289+0000","last_clean":"2024-09-05T13:18:18.260289+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:18.260289+0000","last_undegraded":"2024-09-05T13:18:18.260289+0000","last_fullsized":"2024-09-05T13:18:18.260289+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11253,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13526","reported_seq":13655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.294410+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:18.294410+0000","last_peered":"2024-09-05T13:18:18.294410+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:18.294410+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412248,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11337,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1566},{"shards":"2","objects":3178}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13320","reported_seq":13390,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.254046+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:18.254046+0000","last_peered":"2024-09-05T13:18:18.254046+0000","last_clean":"2024-09-05T13:18:18.254046+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:18.254046+0000","last_undegraded":"2024-09-05T13:18:18.254046+0000","last_fullsized":"2024-09-05T13:18:18.254046+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31563776,"num_objects":7706,"num_object_clones":0,"num_object_copies":15412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13320,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13361","reported_seq":13496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.986068+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:18.986068+0000","last_peered":"2024-09-05T13:18:18.986068+0000","last_clean":"2024-09-05T13:18:18.986068+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:18.986068+0000","last_undegraded":"2024-09-05T13:18:18.986068+0000","last_fullsized":"2024-09-05T13:18:18.986068+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9901,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13466","reported_seq":13605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.878413+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:18.878413+0000","last_peered":"2024-09-05T13:18:18.878413+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:18.878413+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9921,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238},{"shards":"0,2","objects":1565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13189","reported_seq":17998,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.980464+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:18.980464+0000","last_peered":"2024-09-05T13:18:18.980464+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:18.980464+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30470144,"num_objects":7439,"num_object_clones":0,"num_object_copies":14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":548,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9508,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":9551872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5335},{"shards":"0,2","objects":1556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13458","reported_seq":13590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.950195+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:18.950195+0000","last_peered":"2024-09-05T13:18:18.950195+0000","last_clean":"2024-09-05T13:18:18.950195+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:18.950195+0000","last_undegraded":"2024-09-05T13:18:18.950195+0000","last_fullsized":"2024-09-05T13:18:18.950195+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31318016,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10068,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13400","reported_seq":13535,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.296349+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:18.296349+0000","last_peered":"2024-09-05T13:18:18.296349+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:18.296349+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31510528,"num_objects":7693,"num_object_clones":0,"num_object_copies":15386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11054,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1601},{"shards":"2","objects":3242}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13322","reported_seq":13459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.000211+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:19.000211+0000","last_peered":"2024-09-05T13:18:19.000211+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:19.000211+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31080448,"num_objects":7588,"num_object_clones":0,"num_object_copies":15176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9689,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3134},{"shards":"0,2","objects":1591}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13285","reported_seq":22186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.324702+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:18.324702+0000","last_peered":"2024-09-05T13:18:18.324702+0000","last_clean":"2024-09-05T13:18:18.324702+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:18.324702+0000","last_undegraded":"2024-09-05T13:18:18.324702+0000","last_fullsized":"2024-09-05T13:18:18.324702+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31236096,"num_objects":7626,"num_object_clones":0,"num_object_copies":15252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10880,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13137","reported_seq":13257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.160177+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:18.160177+0000","last_peered":"2024-09-05T13:18:18.160177+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:18.160177+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30494720,"num_objects":7445,"num_object_clones":0,"num_object_copies":14890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10646,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1544},{"shards":"2","objects":3058}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13327","reported_seq":13468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.932535+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:18.932535+0000","last_peered":"2024-09-05T13:18:18.932535+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:18.932535+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30928896,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9478,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3132},{"shards":"0,2","objects":1532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13432","reported_seq":13619,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.030207+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:19.030207+0000","last_peered":"2024-09-05T13:18:19.030207+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:19.030207+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9331,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3228},{"shards":"0,2","objects":1568}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13468","reported_seq":13607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.932989+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:18.932989+0000","last_peered":"2024-09-05T13:18:18.932989+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:18.932989+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31178752,"num_objects":7612,"num_object_clones":0,"num_object_copies":15224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3046},{"shards":"0,2","objects":1640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13496","reported_seq":13636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.009989+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:19.009989+0000","last_peered":"2024-09-05T13:18:19.009989+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:19.009989+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31523299,"num_objects":7697,"num_object_clones":0,"num_object_copies":15394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3132},{"shards":"0,2","objects":1671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":498655739,"num_objects":121744,"num_object_clones":0,"num_object_copies":243488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45216,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":121744,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169443,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6726,"num_bytes_recovered":27533312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1144979456,"data_stored":1144895439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698148,"kb_used_data":387404,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93673692,"statfs":{"total":96636764160,"available":95921860608,"internally_reserved":0,"allocated":396701696,"data_stored":396442139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":825008,"kb_used_data":436576,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93546832,"statfs":{"total":96636764160,"available":95791955968,"internally_reserved":0,"allocated":447053824,"data_stored":446794262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607004,"kb_used_data":296268,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93764836,"statfs":{"total":96636764160,"available":96015192064,"internally_reserved":0,"allocated":303378432,"data_stored":303049764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446357504,"data_stored":446353891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":302956544,"data_stored":302883825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395665408,"data_stored":395657723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:22.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:23.054 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:23.077 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-05T13:18:21.645952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":498655758,"num_objects":121746,"num_object_clones":0,"num_object_copies":243492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45216,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":121746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169445,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6727,"num_bytes_recovered":27533331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2130160,"kb_used_data":1120248,"kb_used_omap":12,"kb_used_meta":1009843,"kb_avail":280985360,"statfs":{"total":289910292480,"available":287729008640,"internally_reserved":0,"allocated":1147133952,"data_stored":1146286165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1034079238},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,6,2,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17920000,"num_objects":-4375,"num_object_clones":0,"num_object_copies":-8750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":65,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":-4375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4375,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005837"},"pg_stats":[{"pgid":"2.f","version":"45'13295","reported_seq":13370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.001888+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:19.001888+0000","last_peered":"2024-09-05T13:18:19.001888+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:19.001888+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13295,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3256},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13243","reported_seq":13348,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.260289+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:18.260289+0000","last_peered":"2024-09-05T13:18:18.260289+0000","last_clean":"2024-09-05T13:18:18.260289+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:18.260289+0000","last_undegraded":"2024-09-05T13:18:18.260289+0000","last_fullsized":"2024-09-05T13:18:18.260289+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30765056,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11253,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13526","reported_seq":13655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.294410+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:18.294410+0000","last_peered":"2024-09-05T13:18:18.294410+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:18.294410+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412248,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11337,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1566},{"shards":"2","objects":3178}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13320","reported_seq":13390,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.254046+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:18.254046+0000","last_peered":"2024-09-05T13:18:18.254046+0000","last_clean":"2024-09-05T13:18:18.254046+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:18.254046+0000","last_undegraded":"2024-09-05T13:18:18.254046+0000","last_fullsized":"2024-09-05T13:18:18.254046+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31563776,"num_objects":7706,"num_object_clones":0,"num_object_copies":15412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13320,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13361","reported_seq":13496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.986068+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:18.986068+0000","last_peered":"2024-09-05T13:18:18.986068+0000","last_clean":"2024-09-05T13:18:18.986068+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:18.986068+0000","last_undegraded":"2024-09-05T13:18:18.986068+0000","last_fullsized":"2024-09-05T13:18:18.986068+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9901,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13466","reported_seq":13605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.878413+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:18.878413+0000","last_peered":"2024-09-05T13:18:18.878413+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:18.878413+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9921,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238},{"shards":"0,2","objects":1565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13189","reported_seq":17998,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.980464+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:18.980464+0000","last_peered":"2024-09-05T13:18:18.980464+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:18.980464+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30470144,"num_objects":7439,"num_object_clones":0,"num_object_copies":14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":548,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9508,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":9551872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5335},{"shards":"0,2","objects":1556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13458","reported_seq":13590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.950195+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:18.950195+0000","last_peered":"2024-09-05T13:18:18.950195+0000","last_clean":"2024-09-05T13:18:18.950195+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:18.950195+0000","last_undegraded":"2024-09-05T13:18:18.950195+0000","last_fullsized":"2024-09-05T13:18:18.950195+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31318016,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10068,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13400","reported_seq":13535,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.296349+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:18.296349+0000","last_peered":"2024-09-05T13:18:18.296349+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:18.296349+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31510528,"num_objects":7693,"num_object_clones":0,"num_object_copies":15386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11054,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1601},{"shards":"2","objects":3242}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13322","reported_seq":13459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.000211+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:19.000211+0000","last_peered":"2024-09-05T13:18:19.000211+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:19.000211+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31080448,"num_objects":7588,"num_object_clones":0,"num_object_copies":15176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9689,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3134},{"shards":"0,2","objects":1591}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13285","reported_seq":22186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.324702+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:18.324702+0000","last_peered":"2024-09-05T13:18:18.324702+0000","last_clean":"2024-09-05T13:18:18.324702+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:18.324702+0000","last_undegraded":"2024-09-05T13:18:18.324702+0000","last_fullsized":"2024-09-05T13:18:18.324702+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31236096,"num_objects":7626,"num_object_clones":0,"num_object_copies":15252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10880,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13137","reported_seq":13257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.160177+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:18.160177+0000","last_peered":"2024-09-05T13:18:18.160177+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:18.160177+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30494720,"num_objects":7445,"num_object_clones":0,"num_object_copies":14890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10646,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1544},{"shards":"2","objects":3058}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13327","reported_seq":13468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.932535+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:18.932535+0000","last_peered":"2024-09-05T13:18:18.932535+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:18.932535+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30928896,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9478,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3132},{"shards":"0,2","objects":1532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13432","reported_seq":13619,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.030207+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:19.030207+0000","last_peered":"2024-09-05T13:18:19.030207+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:19.030207+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9331,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3228},{"shards":"0,2","objects":1568}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13468","reported_seq":13607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.932989+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:18.932989+0000","last_peered":"2024-09-05T13:18:18.932989+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:18.932989+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31178752,"num_objects":7612,"num_object_clones":0,"num_object_copies":15224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3046},{"shards":"0,2","objects":1640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13496","reported_seq":13636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.009989+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:19.009989+0000","last_peered":"2024-09-05T13:18:19.009989+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:19.009989+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31523299,"num_objects":7697,"num_object_clones":0,"num_object_copies":15394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3132},{"shards":"0,2","objects":1671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":498655739,"num_objects":121744,"num_object_clones":0,"num_object_copies":243488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45216,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":121744,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169443,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6726,"num_bytes_recovered":27533312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1144979456,"data_stored":1144895439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698148,"kb_used_data":387404,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93673692,"statfs":{"total":96636764160,"available":95921860608,"internally_reserved":0,"allocated":396701696,"data_stored":396442139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":825008,"kb_used_data":436576,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93546832,"statfs":{"total":96636764160,"available":95791955968,"internally_reserved":0,"allocated":447053824,"data_stored":446794262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607004,"kb_used_data":296268,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93764836,"statfs":{"total":96636764160,"available":96015192064,"internally_reserved":0,"allocated":303378432,"data_stored":303049764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446357504,"data_stored":446353891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":302956544,"data_stored":302883825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395665408,"data_stored":395657723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:23.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:23.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13295", 'reported_seq': 13370, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:19.001888+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:19.001888+0000', 'last_peered': '2024-09-05T13:18:19.001888+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:19.001888+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31444992, 'num_objects': 7677, 'num_object_clones': 0, 'num_object_copies': 15354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7677, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13295, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3256}, {'shards': '0,2', 'objects': 1611}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:23.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:23.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'13526", 'reported_seq': 13655, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:18.294410+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:18.294410+0000', 'last_peered': '2024-09-05T13:18:18.294410+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:18.294410+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31412248, 'num_objects': 7670, 'num_object_clones': 0, 'num_object_copies': 15340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7670, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11337, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1566}, {'shards': '2', 'objects': 3178}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:23.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:23.081 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'13466", 'reported_seq': 13605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:18.878413+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:18.878413+0000', 'last_peered': '2024-09-05T13:18:18.878413+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:18.878413+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31494144, 'num_objects': 7689, 'num_object_clones': 0, 'num_object_copies': 15378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7689, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9921, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3238}, {'shards': '0,2', 'objects': 1565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:23.081 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:23.081 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'13189", 'reported_seq': 17998, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:18.980464+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:18.980464+0000', 'last_peered': '2024-09-05T13:18:18.980464+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:18.980464+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30470144, 'num_objects': 7439, 'num_object_clones': 0, 'num_object_copies': 14878, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 548, 'num_objects_misplaced': 1556, 'num_objects_unfound': 0, 'num_objects_dirty': 7439, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9508, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2334, 'num_bytes_recovered': 9551872, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5335}, {'shards': '0,2', 'objects': 1556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:23.081 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:23.082 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'13400", 'reported_seq': 13535, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:18.296349+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:18.296349+0000', 'last_peered': '2024-09-05T13:18:18.296349+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:18.296349+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31510528, 'num_objects': 7693, 'num_object_clones': 0, 'num_object_copies': 15386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7693, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11054, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1601}, {'shards': '2', 'objects': 3242}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:23.082 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:23.082 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'13322", 'reported_seq': 13459, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:19.000211+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:19.000211+0000', 'last_peered': '2024-09-05T13:18:19.000211+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:19.000211+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31080448, 'num_objects': 7588, 'num_object_clones': 0, 'num_object_copies': 15176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9689, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3134}, {'shards': '0,2', 'objects': 1591}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:23.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:23.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'13137", 'reported_seq': 13257, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:18.160177+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:18.160177+0000', 'last_peered': '2024-09-05T13:18:18.160177+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:18.160177+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30494720, 'num_objects': 7445, 'num_object_clones': 0, 'num_object_copies': 14890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10646, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1544}, {'shards': '2', 'objects': 3058}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:23.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:23.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'13327", 'reported_seq': 13468, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:18.932535+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:18.932535+0000', 'last_peered': '2024-09-05T13:18:18.932535+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:18.932535+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30928896, 'num_objects': 7551, 'num_object_clones': 0, 'num_object_copies': 15102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9478, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3132}, {'shards': '0,2', 'objects': 1532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:23.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:23.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'13432", 'reported_seq': 13619, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:19.030207+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:19.030207+0000', 'last_peered': '2024-09-05T13:18:19.030207+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:19.030207+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31424512, 'num_objects': 7672, 'num_object_clones': 0, 'num_object_copies': 15344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9331, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3228}, {'shards': '0,2', 'objects': 1568}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:23.084 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:23.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'13468", 'reported_seq': 13607, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:18.932989+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:18.932989+0000', 'last_peered': '2024-09-05T13:18:18.932989+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:18.932989+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31178752, 'num_objects': 7612, 'num_object_clones': 0, 'num_object_copies': 15224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9598, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3046}, {'shards': '0,2', 'objects': 1640}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:23.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:23.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'13496", 'reported_seq': 13636, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:19.009989+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:19.009989+0000', 'last_peered': '2024-09-05T13:18:19.009989+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:19.009989+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31523299, 'num_objects': 7697, 'num_object_clones': 0, 'num_object_copies': 15394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10164, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3132}, {'shards': '0,2', 'objects': 1671}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:23.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:23.789 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:23.790 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45216/243492 objects degraded (18.570%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":430,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":121746,"data_bytes":498655758,"bytes_used":2181283840,"bytes_avail":287729008640,"bytes_total":289910292480,"degraded_objects":45216,"degraded_total":243492,"degraded_ratio":0.18569809275048052,"misplaced_objects":1556,"misplaced_total":243492,"misplaced_ratio":0.006390353687184794,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":73692,"recovering_keys_per_sec":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":364},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:23.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:23.820 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:24.382 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:24.385 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:24.420 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-05T13:18:23.646557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495342094,"num_objects":120937,"num_object_clones":0,"num_object_copies":241874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45216,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":120937,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170254,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6727,"num_bytes_recovered":27533331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160937,"ondisk_log_size":160937,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2100484,"kb_used_data":1115540,"kb_used_omap":12,"kb_used_meta":984883,"kb_avail":281015036,"statfs":{"total":289910292480,"available":287759396864,"internally_reserved":0,"allocated":1142312960,"data_stored":1141465173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1008520198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,6,7,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21233664,"num_objects":-5184,"num_object_clones":0,"num_object_copies":-10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":65,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":-5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5184,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005741"},"pg_stats":[{"pgid":"2.f","version":"45'13295","reported_seq":13370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.001888+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:19.001888+0000","last_peered":"2024-09-05T13:18:19.001888+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:19.001888+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31444992,"num_objects":7677,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13295,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3256},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13375","reported_seq":13480,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.167460+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:23.167460+0000","last_peered":"2024-09-05T13:18:23.167460+0000","last_clean":"2024-09-05T13:18:23.167460+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:23.167460+0000","last_undegraded":"2024-09-05T13:18:23.167460+0000","last_fullsized":"2024-09-05T13:18:23.167460+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11385,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13670","reported_seq":13799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.280033+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:23.280033+0000","last_peered":"2024-09-05T13:18:23.280033+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:23.280033+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30822424,"num_objects":7526,"num_object_clones":0,"num_object_copies":15052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11481,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1422},{"shards":"2","objects":3034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13448","reported_seq":13518,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.324675+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:23.324675+0000","last_peered":"2024-09-05T13:18:23.324675+0000","last_clean":"2024-09-05T13:18:23.324675+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:23.324675+0000","last_undegraded":"2024-09-05T13:18:23.324675+0000","last_fullsized":"2024-09-05T13:18:23.324675+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13361","reported_seq":13496,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.986068+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:18.986068+0000","last_peered":"2024-09-05T13:18:18.986068+0000","last_clean":"2024-09-05T13:18:18.986068+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:18.986068+0000","last_undegraded":"2024-09-05T13:18:18.986068+0000","last_fullsized":"2024-09-05T13:18:18.986068+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9901,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13466","reported_seq":13605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.878413+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:18.878413+0000","last_peered":"2024-09-05T13:18:18.878413+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:18.878413+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31494144,"num_objects":7689,"num_object_clones":0,"num_object_copies":15378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9921,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238},{"shards":"0,2","objects":1565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13189","reported_seq":17998,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.980464+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:18.980464+0000","last_peered":"2024-09-05T13:18:18.980464+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:18.980464+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30470144,"num_objects":7439,"num_object_clones":0,"num_object_copies":14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":548,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9508,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":9551872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5335},{"shards":"0,2","objects":1556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13458","reported_seq":13590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:18.950195+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:18.950195+0000","last_peered":"2024-09-05T13:18:18.950195+0000","last_clean":"2024-09-05T13:18:18.950195+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:18.950195+0000","last_undegraded":"2024-09-05T13:18:18.950195+0000","last_fullsized":"2024-09-05T13:18:18.950195+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31318016,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10068,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13541","reported_seq":13676,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.257619+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:23.257619+0000","last_peered":"2024-09-05T13:18:23.257619+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:23.257619+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30932992,"num_objects":7552,"num_object_clones":0,"num_object_copies":15104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11195,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1460},{"shards":"2","objects":3101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13322","reported_seq":13459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.000211+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:19.000211+0000","last_peered":"2024-09-05T13:18:19.000211+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:19.000211+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31080448,"num_objects":7588,"num_object_clones":0,"num_object_copies":15176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9689,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3134},{"shards":"0,2","objects":1591}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13412","reported_seq":22313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.275931+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:23.275931+0000","last_peered":"2024-09-05T13:18:23.275931+0000","last_clean":"2024-09-05T13:18:23.275931+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:23.275931+0000","last_undegraded":"2024-09-05T13:18:23.275931+0000","last_fullsized":"2024-09-05T13:18:23.275931+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30715904,"num_objects":7499,"num_object_clones":0,"num_object_copies":14998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11007,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13274","reported_seq":13398,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.328841+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:23.328841+0000","last_peered":"2024-09-05T13:18:23.328841+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:23.328841+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29933568,"num_objects":7308,"num_object_clones":0,"num_object_copies":14616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10783,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1407},{"shards":"2","objects":2921}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13327","reported_seq":13468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.932535+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:18.932535+0000","last_peered":"2024-09-05T13:18:18.932535+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:18.932535+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30928896,"num_objects":7551,"num_object_clones":0,"num_object_copies":15102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9478,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3132},{"shards":"0,2","objects":1532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13432","reported_seq":13619,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.030207+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:19.030207+0000","last_peered":"2024-09-05T13:18:19.030207+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:19.030207+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9331,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3228},{"shards":"0,2","objects":1568}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13468","reported_seq":13607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:18.932989+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:18.932989+0000","last_peered":"2024-09-05T13:18:18.932989+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:18.932989+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31178752,"num_objects":7612,"num_object_clones":0,"num_object_copies":15224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3046},{"shards":"0,2","objects":1640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13496","reported_seq":13636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:19.009989+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:19.009989+0000","last_peered":"2024-09-05T13:18:19.009989+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:19.009989+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31523299,"num_objects":7697,"num_object_clones":0,"num_object_copies":15394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3132},{"shards":"0,2","objects":1671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495342075,"num_objects":120935,"num_object_clones":0,"num_object_copies":241870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45216,"num_objects_misplaced":1556,"num_objects_unfound":0,"num_objects_dirty":120935,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170252,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6726,"num_bytes_recovered":27533312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1140109312,"data_stored":1140025295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160934,"ondisk_log_size":160934,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681480,"kb_used_data":383224,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93690360,"statfs":{"total":96636764160,"available":95938928640,"internally_reserved":0,"allocated":392421376,"data_stored":392161819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":825008,"kb_used_data":436576,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93546832,"statfs":{"total":96636764160,"available":95791955968,"internally_reserved":0,"allocated":447053824,"data_stored":446794262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593996,"kb_used_data":295740,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93777844,"statfs":{"total":96636764160,"available":96028512256,"internally_reserved":0,"allocated":302837760,"data_stored":302509092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446357504,"data_stored":446353891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":302419968,"data_stored":302347249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391331840,"data_stored":391324155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:27.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:27.982 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:28.002 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-05T13:18:27.647785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489615886,"num_objects":119539,"num_object_clones":0,"num_object_copies":239078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45256,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":119539,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171652,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6833,"num_bytes_recovered":27967507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110152,"kb_used_data":1108568,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281005368,"statfs":{"total":289910292480,"available":287749496832,"internally_reserved":0,"allocated":1135173632,"data_stored":1134325845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1025559558},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,7,12,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17903616,"num_objects":-4371,"num_object_clones":0,"num_object_copies":-8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":73,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":-4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003902"},"pg_stats":[{"pgid":"2.f","version":"45'13425","reported_seq":13500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.985983+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:23.985983+0000","last_peered":"2024-09-05T13:18:23.985983+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:23.985983+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30912512,"num_objects":7547,"num_object_clones":0,"num_object_copies":15094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13425,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":1481}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13375","reported_seq":13480,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.167460+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:23.167460+0000","last_peered":"2024-09-05T13:18:23.167460+0000","last_clean":"2024-09-05T13:18:23.167460+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:23.167460+0000","last_undegraded":"2024-09-05T13:18:23.167460+0000","last_fullsized":"2024-09-05T13:18:23.167460+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11385,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13670","reported_seq":13799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.280033+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:23.280033+0000","last_peered":"2024-09-05T13:18:23.280033+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:23.280033+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30822424,"num_objects":7526,"num_object_clones":0,"num_object_copies":15052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11481,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1422},{"shards":"2","objects":3034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13448","reported_seq":13518,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.324675+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:23.324675+0000","last_peered":"2024-09-05T13:18:23.324675+0000","last_clean":"2024-09-05T13:18:23.324675+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:23.324675+0000","last_undegraded":"2024-09-05T13:18:23.324675+0000","last_fullsized":"2024-09-05T13:18:23.324675+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13492","reported_seq":13627,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:24.023530+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:24.023530+0000","last_peered":"2024-09-05T13:18:24.023530+0000","last_clean":"2024-09-05T13:18:24.023530+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:24.023530+0000","last_undegraded":"2024-09-05T13:18:24.023530+0000","last_fullsized":"2024-09-05T13:18:24.023530+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10032,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13594","reported_seq":13733,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.074330+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:24.074330+0000","last_peered":"2024-09-05T13:18:24.074330+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:24.074330+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969856,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10049,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3110},{"shards":"0,2","objects":1437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13341","reported_seq":18362,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.081959+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:24.081959+0000","last_peered":"2024-09-05T13:18:24.081959+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:24.081959+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29847552,"num_objects":7287,"num_object_clones":0,"num_object_copies":14574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":588,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":7287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9660,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2440,"num_bytes_recovered":9986048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5289},{"shards":"0,2","objects":1410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13596","reported_seq":13728,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:24.061048+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:24.061048+0000","last_peered":"2024-09-05T13:18:24.061048+0000","last_clean":"2024-09-05T13:18:24.061048+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:24.061048+0000","last_undegraded":"2024-09-05T13:18:24.061048+0000","last_fullsized":"2024-09-05T13:18:24.061048+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10206,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13541","reported_seq":13676,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.257619+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:23.257619+0000","last_peered":"2024-09-05T13:18:23.257619+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:23.257619+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30932992,"num_objects":7552,"num_object_clones":0,"num_object_copies":15104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11195,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1460},{"shards":"2","objects":3101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13469","reported_seq":13607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.031458+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:24.031458+0000","last_peered":"2024-09-05T13:18:24.031458+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:24.031458+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9836,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2987},{"shards":"0,2","objects":1444}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13412","reported_seq":22313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.275931+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:23.275931+0000","last_peered":"2024-09-05T13:18:23.275931+0000","last_clean":"2024-09-05T13:18:23.275931+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:23.275931+0000","last_undegraded":"2024-09-05T13:18:23.275931+0000","last_fullsized":"2024-09-05T13:18:23.275931+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30715904,"num_objects":7499,"num_object_clones":0,"num_object_copies":14998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11007,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13274","reported_seq":13398,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.328841+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:23.328841+0000","last_peered":"2024-09-05T13:18:23.328841+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:23.328841+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29933568,"num_objects":7308,"num_object_clones":0,"num_object_copies":14616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10783,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1407},{"shards":"2","objects":2921}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13458","reported_seq":13597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.084238+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:24.084238+0000","last_peered":"2024-09-05T13:18:24.084238+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:24.084238+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30392320,"num_objects":7420,"num_object_clones":0,"num_object_copies":14840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9609,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3001},{"shards":"0,2","objects":1401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13571","reported_seq":13759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.065085+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:24.065085+0000","last_peered":"2024-09-05T13:18:24.065085+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:24.065085+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30855168,"num_objects":7533,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9470,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3089},{"shards":"0,2","objects":1429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13627","reported_seq":13766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.083935+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:24.083935+0000","last_peered":"2024-09-05T13:18:24.083935+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:24.083935+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30527488,"num_objects":7453,"num_object_clones":0,"num_object_copies":14906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9757,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2887},{"shards":"0,2","objects":1481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13639","reported_seq":13779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.967947+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:23.967947+0000","last_peered":"2024-09-05T13:18:23.967947+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:23.967947+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30937571,"num_objects":7554,"num_object_clones":0,"num_object_copies":15108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10307,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2989},{"shards":"0,2","objects":1528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489615867,"num_objects":119537,"num_object_clones":0,"num_object_copies":239074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45256,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":119537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171650,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6832,"num_bytes_recovered":27967488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1132797952,"data_stored":1132713935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681480,"kb_used_data":383224,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93690360,"statfs":{"total":96636764160,"available":95938928640,"internally_reserved":0,"allocated":392421376,"data_stored":392161819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826888,"kb_used_data":430136,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93544952,"statfs":{"total":96636764160,"available":95790030848,"internally_reserved":0,"allocated":440459264,"data_stored":440199702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601784,"kb_used_data":295208,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93770056,"statfs":{"total":96636764160,"available":96020537344,"internally_reserved":0,"allocated":302292992,"data_stored":301964324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":439607296,"data_stored":439603683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301858816,"data_stored":301786097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391331840,"data_stored":391324155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:28.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:28.551 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:28.551 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:28.579 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-05T13:18:27.647785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489615886,"num_objects":119539,"num_object_clones":0,"num_object_copies":239078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45256,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":119539,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171652,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6833,"num_bytes_recovered":27967507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110152,"kb_used_data":1108568,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281005368,"statfs":{"total":289910292480,"available":287749496832,"internally_reserved":0,"allocated":1135173632,"data_stored":1134325845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1025559558},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,7,12,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17903616,"num_objects":-4371,"num_object_clones":0,"num_object_copies":-8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":73,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":-4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003902"},"pg_stats":[{"pgid":"2.f","version":"45'13425","reported_seq":13500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.985983+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:23.985983+0000","last_peered":"2024-09-05T13:18:23.985983+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:23.985983+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30912512,"num_objects":7547,"num_object_clones":0,"num_object_copies":15094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13425,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":1481}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13375","reported_seq":13480,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.167460+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:23.167460+0000","last_peered":"2024-09-05T13:18:23.167460+0000","last_clean":"2024-09-05T13:18:23.167460+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:23.167460+0000","last_undegraded":"2024-09-05T13:18:23.167460+0000","last_fullsized":"2024-09-05T13:18:23.167460+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11385,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13670","reported_seq":13799,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.280033+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:23.280033+0000","last_peered":"2024-09-05T13:18:23.280033+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:23.280033+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30822424,"num_objects":7526,"num_object_clones":0,"num_object_copies":15052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11481,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1422},{"shards":"2","objects":3034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13448","reported_seq":13518,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.324675+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:23.324675+0000","last_peered":"2024-09-05T13:18:23.324675+0000","last_clean":"2024-09-05T13:18:23.324675+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:23.324675+0000","last_undegraded":"2024-09-05T13:18:23.324675+0000","last_fullsized":"2024-09-05T13:18:23.324675+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13492","reported_seq":13627,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:24.023530+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:24.023530+0000","last_peered":"2024-09-05T13:18:24.023530+0000","last_clean":"2024-09-05T13:18:24.023530+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:24.023530+0000","last_undegraded":"2024-09-05T13:18:24.023530+0000","last_fullsized":"2024-09-05T13:18:24.023530+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10032,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13594","reported_seq":13733,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.074330+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:24.074330+0000","last_peered":"2024-09-05T13:18:24.074330+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:24.074330+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969856,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10049,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3110},{"shards":"0,2","objects":1437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13341","reported_seq":18362,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.081959+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:24.081959+0000","last_peered":"2024-09-05T13:18:24.081959+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:24.081959+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29847552,"num_objects":7287,"num_object_clones":0,"num_object_copies":14574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":588,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":7287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9660,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2440,"num_bytes_recovered":9986048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5289},{"shards":"0,2","objects":1410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13596","reported_seq":13728,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:24.061048+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:24.061048+0000","last_peered":"2024-09-05T13:18:24.061048+0000","last_clean":"2024-09-05T13:18:24.061048+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:24.061048+0000","last_undegraded":"2024-09-05T13:18:24.061048+0000","last_fullsized":"2024-09-05T13:18:24.061048+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10206,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13541","reported_seq":13676,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.257619+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:23.257619+0000","last_peered":"2024-09-05T13:18:23.257619+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:23.257619+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30932992,"num_objects":7552,"num_object_clones":0,"num_object_copies":15104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11195,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1460},{"shards":"2","objects":3101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13469","reported_seq":13607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.031458+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:24.031458+0000","last_peered":"2024-09-05T13:18:24.031458+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:24.031458+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30478336,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9836,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2987},{"shards":"0,2","objects":1444}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13412","reported_seq":22313,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:23.275931+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:23.275931+0000","last_peered":"2024-09-05T13:18:23.275931+0000","last_clean":"2024-09-05T13:18:23.275931+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:23.275931+0000","last_undegraded":"2024-09-05T13:18:23.275931+0000","last_fullsized":"2024-09-05T13:18:23.275931+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30715904,"num_objects":7499,"num_object_clones":0,"num_object_copies":14998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11007,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13274","reported_seq":13398,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.328841+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:23.328841+0000","last_peered":"2024-09-05T13:18:23.328841+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:23.328841+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29933568,"num_objects":7308,"num_object_clones":0,"num_object_copies":14616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10783,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1407},{"shards":"2","objects":2921}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13458","reported_seq":13597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.084238+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:24.084238+0000","last_peered":"2024-09-05T13:18:24.084238+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:24.084238+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30392320,"num_objects":7420,"num_object_clones":0,"num_object_copies":14840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9609,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3001},{"shards":"0,2","objects":1401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13571","reported_seq":13759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.065085+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:24.065085+0000","last_peered":"2024-09-05T13:18:24.065085+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:24.065085+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30855168,"num_objects":7533,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9470,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3089},{"shards":"0,2","objects":1429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13627","reported_seq":13766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:24.083935+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:24.083935+0000","last_peered":"2024-09-05T13:18:24.083935+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:24.083935+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30527488,"num_objects":7453,"num_object_clones":0,"num_object_copies":14906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9757,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2887},{"shards":"0,2","objects":1481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13639","reported_seq":13779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:23.967947+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:23.967947+0000","last_peered":"2024-09-05T13:18:23.967947+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:23.967947+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30937571,"num_objects":7554,"num_object_clones":0,"num_object_copies":15108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10307,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2989},{"shards":"0,2","objects":1528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489615867,"num_objects":119537,"num_object_clones":0,"num_object_copies":239074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45256,"num_objects_misplaced":1410,"num_objects_unfound":0,"num_objects_dirty":119537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171650,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6832,"num_bytes_recovered":27967488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1132797952,"data_stored":1132713935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681480,"kb_used_data":383224,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93690360,"statfs":{"total":96636764160,"available":95938928640,"internally_reserved":0,"allocated":392421376,"data_stored":392161819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":826888,"kb_used_data":430136,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93544952,"statfs":{"total":96636764160,"available":95790030848,"internally_reserved":0,"allocated":440459264,"data_stored":440199702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601784,"kb_used_data":295208,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93770056,"statfs":{"total":96636764160,"available":96020537344,"internally_reserved":0,"allocated":302292992,"data_stored":301964324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":439607296,"data_stored":439603683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301858816,"data_stored":301786097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391331840,"data_stored":391324155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:28.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:28.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13425", 'reported_seq': 13500, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:23.985983+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:23.985983+0000', 'last_peered': '2024-09-05T13:18:23.985983+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:23.985983+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30912512, 'num_objects': 7547, 'num_object_clones': 0, 'num_object_copies': 15094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7547, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13425, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3126}, {'shards': '0,2', 'objects': 1481}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:28.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'13670", 'reported_seq': 13799, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:23.280033+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:23.280033+0000', 'last_peered': '2024-09-05T13:18:23.280033+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:23.280033+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30822424, 'num_objects': 7526, 'num_object_clones': 0, 'num_object_copies': 15052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7526, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11481, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1422}, {'shards': '2', 'objects': 3034}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'13594", 'reported_seq': 13733, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:24.074330+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:24.074330+0000', 'last_peered': '2024-09-05T13:18:24.074330+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:24.074330+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30969856, 'num_objects': 7561, 'num_object_clones': 0, 'num_object_copies': 15122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10049, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3110}, {'shards': '0,2', 'objects': 1437}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'13341", 'reported_seq': 18362, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:24.081959+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:24.081959+0000', 'last_peered': '2024-09-05T13:18:24.081959+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:24.081959+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29847552, 'num_objects': 7287, 'num_object_clones': 0, 'num_object_copies': 14574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 588, 'num_objects_misplaced': 1410, 'num_objects_unfound': 0, 'num_objects_dirty': 7287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9660, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2440, 'num_bytes_recovered': 9986048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5289}, {'shards': '0,2', 'objects': 1410}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'13541", 'reported_seq': 13676, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:23.257619+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:23.257619+0000', 'last_peered': '2024-09-05T13:18:23.257619+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:23.257619+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30932992, 'num_objects': 7552, 'num_object_clones': 0, 'num_object_copies': 15104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11195, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1460}, {'shards': '2', 'objects': 3101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:28.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'13469", 'reported_seq': 13607, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:24.031458+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:24.031458+0000', 'last_peered': '2024-09-05T13:18:24.031458+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:24.031458+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30478336, 'num_objects': 7441, 'num_object_clones': 0, 'num_object_copies': 14882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7441, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9836, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2987}, {'shards': '0,2', 'objects': 1444}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:28.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:28.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'13274", 'reported_seq': 13398, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:23.328841+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:23.328841+0000', 'last_peered': '2024-09-05T13:18:23.328841+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:23.328841+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29933568, 'num_objects': 7308, 'num_object_clones': 0, 'num_object_copies': 14616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10783, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1407}, {'shards': '2', 'objects': 2921}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:28.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:28.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'13458", 'reported_seq': 13597, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:24.084238+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:24.084238+0000', 'last_peered': '2024-09-05T13:18:24.084238+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:24.084238+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30392320, 'num_objects': 7420, 'num_object_clones': 0, 'num_object_copies': 14840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9609, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3001}, {'shards': '0,2', 'objects': 1401}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:28.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:28.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'13571", 'reported_seq': 13759, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:24.065085+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:24.065085+0000', 'last_peered': '2024-09-05T13:18:24.065085+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:24.065085+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30855168, 'num_objects': 7533, 'num_object_clones': 0, 'num_object_copies': 15066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9470, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3089}, {'shards': '0,2', 'objects': 1429}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:28.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:28.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'13627", 'reported_seq': 13766, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:24.083935+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:24.083935+0000', 'last_peered': '2024-09-05T13:18:24.083935+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:24.083935+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30527488, 'num_objects': 7453, 'num_object_clones': 0, 'num_object_copies': 14906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7453, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9757, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2887}, {'shards': '0,2', 'objects': 1481}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:28.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:28.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'13639", 'reported_seq': 13779, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:23.967947+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:23.967947+0000', 'last_peered': '2024-09-05T13:18:23.967947+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:23.967947+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30937571, 'num_objects': 7554, 'num_object_clones': 0, 'num_object_copies': 15108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7554, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10307, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2989}, {'shards': '0,2', 'objects': 1528}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:28.583 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:29.259 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:29.260 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45256/239078 objects degraded (18.929%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":436,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119539,"data_bytes":489615886,"bytes_used":2160795648,"bytes_avail":287749496832,"bytes_total":289910292480,"degraded_objects":45256,"degraded_total":239078,"degraded_ratio":0.18929387061963041,"misplaced_objects":1410,"misplaced_total":239078,"misplaced_ratio":0.005897656831661633,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":73021,"recovering_keys_per_sec":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":364},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:29.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:29.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:29.829 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:29.830 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:29.865 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-05T13:18:29.648845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480342542,"num_objects":117275,"num_object_clones":0,"num_object_copies":234550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45282,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":117275,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173916,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6943,"num_bytes_recovered":28418067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107260,"kb_used_data":1097344,"kb_used_omap":12,"kb_used_meta":1009843,"kb_avail":281008260,"statfs":{"total":289910292480,"available":287752458240,"internally_reserved":0,"allocated":1123680256,"data_stored":1122832469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1034079238},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,6,9,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27176960,"num_objects":-6635,"num_object_clones":0,"num_object_copies":-13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":99,"num_objects_misplaced":-423,"num_objects_unfound":0,"num_objects_dirty":-6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":1327104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004296"},"pg_stats":[{"pgid":"2.f","version":"45'13577","reported_seq":13652,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.215705+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:29.215705+0000","last_peered":"2024-09-05T13:18:29.215705+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:29.215705+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13577,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2974},{"shards":"0,2","objects":1329}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13516","reported_seq":13621,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:28.337817+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:28.337817+0000","last_peered":"2024-09-05T13:18:28.337817+0000","last_clean":"2024-09-05T13:18:28.337817+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:28.337817+0000","last_undegraded":"2024-09-05T13:18:28.337817+0000","last_fullsized":"2024-09-05T13:18:28.337817+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11526,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13805","reported_seq":13934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:28.368174+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:28.368174+0000","last_peered":"2024-09-05T13:18:28.368174+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:28.368174+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30269464,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11616,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1287},{"shards":"2","objects":2899}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13597","reported_seq":13667,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:28.293594+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:28.293594+0000","last_peered":"2024-09-05T13:18:28.293594+0000","last_clean":"2024-09-05T13:18:28.293594+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:28.293594+0000","last_undegraded":"2024-09-05T13:18:28.293594+0000","last_fullsized":"2024-09-05T13:18:28.293594+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13597,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13628","reported_seq":13763,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:29.187760+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:29.187760+0000","last_peered":"2024-09-05T13:18:29.187760+0000","last_clean":"2024-09-05T13:18:29.187760+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:29.187760+0000","last_undegraded":"2024-09-05T13:18:29.187760+0000","last_fullsized":"2024-09-05T13:18:29.187760+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29716480,"num_objects":7255,"num_object_clones":0,"num_object_copies":14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10168,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13709","reported_seq":13848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.053063+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:29.053063+0000","last_peered":"2024-09-05T13:18:29.053063+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:29.053063+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2995},{"shards":"0,2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13478","reported_seq":18717,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.206994+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:29.206994+0000","last_peered":"2024-09-05T13:18:29.206994+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:29.206994+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29286400,"num_objects":7150,"num_object_clones":0,"num_object_copies":14300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":7150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9797,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2550,"num_bytes_recovered":10436608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13741","reported_seq":13872,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:29.154856+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:29.154856+0000","last_peered":"2024-09-05T13:18:29.154856+0000","last_clean":"2024-09-05T13:18:29.154856+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:29.154856+0000","last_undegraded":"2024-09-05T13:18:29.154856+0000","last_fullsized":"2024-09-05T13:18:29.154856+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30158848,"num_objects":7363,"num_object_clones":0,"num_object_copies":14726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10351,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13694","reported_seq":13829,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:28.331744+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:28.331744+0000","last_peered":"2024-09-05T13:18:28.331744+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:28.331744+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30306304,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11348,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1307},{"shards":"2","objects":2948}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13604","reported_seq":13743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.169917+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:29.169917+0000","last_peered":"2024-09-05T13:18:29.169917+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:29.169917+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29925376,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2852},{"shards":"0,2","objects":1309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13550","reported_seq":22451,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:28.382675+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:28.382675+0000","last_peered":"2024-09-05T13:18:28.382675+0000","last_clean":"2024-09-05T13:18:28.382675+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:28.382675+0000","last_undegraded":"2024-09-05T13:18:28.382675+0000","last_fullsized":"2024-09-05T13:18:28.382675+0000","mapping_epoch":44,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30150656,"num_objects":7361,"num_object_clones":0,"num_object_copies":14722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11145,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13419","reported_seq":13543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:28.294541+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:28.294541+0000","last_peered":"2024-09-05T13:18:28.294541+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:28.294541+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29339648,"num_objects":7163,"num_object_clones":0,"num_object_copies":14326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10928,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1262},{"shards":"2","objects":2776}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13576","reported_seq":13717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.146376+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:29.146376+0000","last_peered":"2024-09-05T13:18:29.146376+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:29.146376+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29908992,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2883},{"shards":"0,2","objects":1283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13729","reported_seq":13917,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.171762+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:29.171762+0000","last_peered":"2024-09-05T13:18:29.171762+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:29.171762+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9628,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2931},{"shards":"0,2","objects":1271}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13777","reported_seq":13913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.196673+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:29.196673+0000","last_peered":"2024-09-05T13:18:29.196673+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:29.196673+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29913088,"num_objects":7303,"num_object_clones":0,"num_object_copies":14606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9907,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2737},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13796","reported_seq":13937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.239833+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:29.239833+0000","last_peered":"2024-09-05T13:18:29.239833+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:29.239833+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294499,"num_objects":7397,"num_object_clones":0,"num_object_copies":14794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2832},{"shards":"0,2","objects":1371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480342523,"num_objects":117273,"num_object_clones":0,"num_object_copies":234546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45282,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":117273,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173914,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6942,"num_bytes_recovered":28418048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1121153024,"data_stored":1121069007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681160,"kb_used_data":378732,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93690680,"statfs":{"total":96636764160,"available":95939256320,"internally_reserved":0,"allocated":387821568,"data_stored":387562011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824316,"kb_used_data":423404,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93547524,"statfs":{"total":96636764160,"available":95792664576,"internally_reserved":0,"allocated":433565696,"data_stored":433306134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601784,"kb_used_data":295208,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93770056,"statfs":{"total":96636764160,"available":96020537344,"internally_reserved":0,"allocated":302292992,"data_stored":301964324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432615424,"data_stored":432611811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301858816,"data_stored":301786097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386678784,"data_stored":386671099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:32.868 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:33.498 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:33.499 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:33.531 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-05T13:18:31.649618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480342542,"num_objects":117275,"num_object_clones":0,"num_object_copies":234550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45282,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":117275,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173916,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6943,"num_bytes_recovered":28418067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107260,"kb_used_data":1097344,"kb_used_omap":12,"kb_used_meta":1009843,"kb_avail":281008260,"statfs":{"total":289910292480,"available":287752458240,"internally_reserved":0,"allocated":1123680256,"data_stored":1122832469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1034079238},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,6,9,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18313216,"num_objects":-4471,"num_object_clones":0,"num_object_copies":-8942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-4471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004409"},"pg_stats":[{"pgid":"2.f","version":"45'13577","reported_seq":13652,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.215705+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:29.215705+0000","last_peered":"2024-09-05T13:18:29.215705+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:29.215705+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13577,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2974},{"shards":"0,2","objects":1329}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13516","reported_seq":13621,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:28.337817+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:28.337817+0000","last_peered":"2024-09-05T13:18:28.337817+0000","last_clean":"2024-09-05T13:18:28.337817+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:28.337817+0000","last_undegraded":"2024-09-05T13:18:28.337817+0000","last_fullsized":"2024-09-05T13:18:28.337817+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11526,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13805","reported_seq":13934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:28.368174+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:28.368174+0000","last_peered":"2024-09-05T13:18:28.368174+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:28.368174+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30269464,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11616,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1287},{"shards":"2","objects":2899}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13597","reported_seq":13667,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:28.293594+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:28.293594+0000","last_peered":"2024-09-05T13:18:28.293594+0000","last_clean":"2024-09-05T13:18:28.293594+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:28.293594+0000","last_undegraded":"2024-09-05T13:18:28.293594+0000","last_fullsized":"2024-09-05T13:18:28.293594+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13597,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13628","reported_seq":13763,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:29.187760+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:29.187760+0000","last_peered":"2024-09-05T13:18:29.187760+0000","last_clean":"2024-09-05T13:18:29.187760+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:29.187760+0000","last_undegraded":"2024-09-05T13:18:29.187760+0000","last_fullsized":"2024-09-05T13:18:29.187760+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29716480,"num_objects":7255,"num_object_clones":0,"num_object_copies":14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10168,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13709","reported_seq":13848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.053063+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:29.053063+0000","last_peered":"2024-09-05T13:18:29.053063+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:29.053063+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2995},{"shards":"0,2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13478","reported_seq":18717,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.206994+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:29.206994+0000","last_peered":"2024-09-05T13:18:29.206994+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:29.206994+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29286400,"num_objects":7150,"num_object_clones":0,"num_object_copies":14300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":7150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9797,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2550,"num_bytes_recovered":10436608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13741","reported_seq":13872,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:29.154856+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:29.154856+0000","last_peered":"2024-09-05T13:18:29.154856+0000","last_clean":"2024-09-05T13:18:29.154856+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:29.154856+0000","last_undegraded":"2024-09-05T13:18:29.154856+0000","last_fullsized":"2024-09-05T13:18:29.154856+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30158848,"num_objects":7363,"num_object_clones":0,"num_object_copies":14726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10351,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13694","reported_seq":13829,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:28.331744+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:28.331744+0000","last_peered":"2024-09-05T13:18:28.331744+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:28.331744+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30306304,"num_objects":7399,"num_object_clones":0,"num_object_copies":14798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11348,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1307},{"shards":"2","objects":2948}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.198248+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:09.198248+0000","last_peered":"2024-09-05T13:16:09.198248+0000","last_clean":"2024-09-05T13:16:09.198248+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:09.198248+0000","last_undegraded":"2024-09-05T13:16:09.198248+0000","last_fullsized":"2024-09-05T13:16:09.198248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13604","reported_seq":13743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.169917+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:29.169917+0000","last_peered":"2024-09-05T13:18:29.169917+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:29.169917+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29925376,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2852},{"shards":"0,2","objects":1309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13550","reported_seq":22451,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:28.382675+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:28.382675+0000","last_peered":"2024-09-05T13:18:28.382675+0000","last_clean":"2024-09-05T13:18:28.382675+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:28.382675+0000","last_undegraded":"2024-09-05T13:18:28.382675+0000","last_fullsized":"2024-09-05T13:18:28.382675+0000","mapping_epoch":44,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30150656,"num_objects":7361,"num_object_clones":0,"num_object_copies":14722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11145,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13419","reported_seq":13543,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:28.294541+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:28.294541+0000","last_peered":"2024-09-05T13:18:28.294541+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:28.294541+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29339648,"num_objects":7163,"num_object_clones":0,"num_object_copies":14326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10928,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1262},{"shards":"2","objects":2776}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13576","reported_seq":13717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.146376+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:29.146376+0000","last_peered":"2024-09-05T13:18:29.146376+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:29.146376+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29908992,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2883},{"shards":"0,2","objects":1283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13729","reported_seq":13917,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.171762+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:29.171762+0000","last_peered":"2024-09-05T13:18:29.171762+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:29.171762+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9628,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2931},{"shards":"0,2","objects":1271}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.199296+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:09.199296+0000","last_peered":"2024-09-05T13:16:09.199296+0000","last_clean":"2024-09-05T13:16:09.199296+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:09.199296+0000","last_undegraded":"2024-09-05T13:16:09.199296+0000","last_fullsized":"2024-09-05T13:16:09.199296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13777","reported_seq":13913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.196673+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:29.196673+0000","last_peered":"2024-09-05T13:18:29.196673+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:29.196673+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29913088,"num_objects":7303,"num_object_clones":0,"num_object_copies":14606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9907,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2737},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13796","reported_seq":13937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.239833+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:29.239833+0000","last_peered":"2024-09-05T13:18:29.239833+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:29.239833+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294499,"num_objects":7397,"num_object_clones":0,"num_object_copies":14794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2832},{"shards":"0,2","objects":1371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480342523,"num_objects":117273,"num_object_clones":0,"num_object_copies":234546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45282,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":117273,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173914,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6942,"num_bytes_recovered":28418048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1121153024,"data_stored":1121069007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681160,"kb_used_data":378732,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93690680,"statfs":{"total":96636764160,"available":95939256320,"internally_reserved":0,"allocated":387821568,"data_stored":387562011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824316,"kb_used_data":423404,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93547524,"statfs":{"total":96636764160,"available":95792664576,"internally_reserved":0,"allocated":433565696,"data_stored":433306134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601784,"kb_used_data":295208,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93770056,"statfs":{"total":96636764160,"available":96020537344,"internally_reserved":0,"allocated":302292992,"data_stored":301964324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432615424,"data_stored":432611811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301858816,"data_stored":301786097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386678784,"data_stored":386671099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:33.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:34.082 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:34.085 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:34.114 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-05T13:18:33.650433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":476901902,"num_objects":116435,"num_object_clones":0,"num_object_copies":232870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45282,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":116435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174756,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6943,"num_bytes_recovered":28418067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110504,"kb_used_data":1092280,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281005016,"statfs":{"total":289910292480,"available":287749136384,"internally_reserved":0,"allocated":1118494720,"data_stored":1117646933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1042598918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,6,8,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21753856,"num_objects":-5311,"num_object_clones":0,"num_object_copies":-10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-5311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5311,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004480"},"pg_stats":[{"pgid":"2.f","version":"45'13577","reported_seq":13652,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.215705+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:29.215705+0000","last_peered":"2024-09-05T13:18:29.215705+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:29.215705+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13577,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2974},{"shards":"0,2","objects":1329}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13658","reported_seq":13763,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.320747+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:33.320747+0000","last_peered":"2024-09-05T13:18:33.320747+0000","last_clean":"2024-09-05T13:18:33.320747+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:33.320747+0000","last_undegraded":"2024-09-05T13:18:33.320747+0000","last_fullsized":"2024-09-05T13:18:33.320747+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11668,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13951","reported_seq":14080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.203915+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:33.203915+0000","last_peered":"2024-09-05T13:18:33.203915+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:33.203915+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671448,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11762,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1141},{"shards":"2","objects":2753}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13732","reported_seq":13802,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.340867+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:33.340867+0000","last_peered":"2024-09-05T13:18:33.340867+0000","last_clean":"2024-09-05T13:18:33.340867+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:33.340867+0000","last_undegraded":"2024-09-05T13:18:33.340867+0000","last_fullsized":"2024-09-05T13:18:33.340867+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13732,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13628","reported_seq":13763,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:29.187760+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:29.187760+0000","last_peered":"2024-09-05T13:18:29.187760+0000","last_clean":"2024-09-05T13:18:29.187760+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:29.187760+0000","last_undegraded":"2024-09-05T13:18:29.187760+0000","last_fullsized":"2024-09-05T13:18:29.187760+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29716480,"num_objects":7255,"num_object_clones":0,"num_object_copies":14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10168,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13709","reported_seq":13848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.053063+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:29.053063+0000","last_peered":"2024-09-05T13:18:29.053063+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:29.053063+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2995},{"shards":"0,2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13478","reported_seq":18717,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.206994+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:29.206994+0000","last_peered":"2024-09-05T13:18:29.206994+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:29.206994+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29286400,"num_objects":7150,"num_object_clones":0,"num_object_copies":14300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":7150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9797,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2550,"num_bytes_recovered":10436608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13741","reported_seq":13872,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:29.154856+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:29.154856+0000","last_peered":"2024-09-05T13:18:29.154856+0000","last_clean":"2024-09-05T13:18:29.154856+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:29.154856+0000","last_undegraded":"2024-09-05T13:18:29.154856+0000","last_fullsized":"2024-09-05T13:18:29.154856+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30158848,"num_objects":7363,"num_object_clones":0,"num_object_copies":14726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10351,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13827","reported_seq":13961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.364577+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:33.364577+0000","last_peered":"2024-09-05T13:18:33.364577+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:33.364577+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29761536,"num_objects":7266,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11481,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1174},{"shards":"2","objects":2815}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13604","reported_seq":13743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.169917+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:29.169917+0000","last_peered":"2024-09-05T13:18:29.169917+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:29.169917+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29925376,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2852},{"shards":"0,2","objects":1309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13702","reported_seq":22604,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.371473+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:33.371473+0000","last_peered":"2024-09-05T13:18:33.371473+0000","last_clean":"2024-09-05T13:18:33.371473+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:33.371473+0000","last_undegraded":"2024-09-05T13:18:33.371473+0000","last_fullsized":"2024-09-05T13:18:33.371473+0000","mapping_epoch":44,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11297,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13551","reported_seq":13675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.364315+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:33.364315+0000","last_peered":"2024-09-05T13:18:33.364315+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:33.364315+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11060,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1130},{"shards":"2","objects":2644}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13576","reported_seq":13717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.146376+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:29.146376+0000","last_peered":"2024-09-05T13:18:29.146376+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:29.146376+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29908992,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2883},{"shards":"0,2","objects":1283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13729","reported_seq":13917,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.171762+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:29.171762+0000","last_peered":"2024-09-05T13:18:29.171762+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:29.171762+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9628,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2931},{"shards":"0,2","objects":1271}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13777","reported_seq":13913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.196673+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:29.196673+0000","last_peered":"2024-09-05T13:18:29.196673+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:29.196673+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29913088,"num_objects":7303,"num_object_clones":0,"num_object_copies":14606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9907,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2737},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13796","reported_seq":13937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.239833+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:29.239833+0000","last_peered":"2024-09-05T13:18:29.239833+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:29.239833+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294499,"num_objects":7397,"num_object_clones":0,"num_object_copies":14794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2832},{"shards":"0,2","objects":1371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":476901883,"num_objects":116433,"num_object_clones":0,"num_object_copies":232866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45282,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":116433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174754,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6942,"num_bytes_recovered":28418048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1115996160,"data_stored":1115912143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680844,"kb_used_data":374268,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93690996,"statfs":{"total":96636764160,"available":95939579904,"internally_reserved":0,"allocated":383250432,"data_stored":382990875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824316,"kb_used_data":423404,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93547524,"statfs":{"total":96636764160,"available":95792664576,"internally_reserved":0,"allocated":433565696,"data_stored":433306134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605344,"kb_used_data":294608,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93766496,"statfs":{"total":96636764160,"available":96016891904,"internally_reserved":0,"allocated":301678592,"data_stored":301349924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432615424,"data_stored":432611811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301252608,"data_stored":301179889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":382128128,"data_stored":382120443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:34.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:34.115 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13577", 'reported_seq': 13652, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:29.215705+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:29.215705+0000', 'last_peered': '2024-09-05T13:18:29.215705+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:29.215705+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30289920, 'num_objects': 7395, 'num_object_clones': 0, 'num_object_copies': 14790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13577, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2974}, {'shards': '0,2', 'objects': 1329}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'13951", 'reported_seq': 14080, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:33.203915+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:33.203915+0000', 'last_peered': '2024-09-05T13:18:33.203915+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:33.203915+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29671448, 'num_objects': 7245, 'num_object_clones': 0, 'num_object_copies': 14490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7245, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11762, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1141}, {'shards': '2', 'objects': 2753}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'13709", 'reported_seq': 13848, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:29.053063+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:29.053063+0000', 'last_peered': '2024-09-05T13:18:29.053063+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:29.053063+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30498816, 'num_objects': 7446, 'num_object_clones': 0, 'num_object_copies': 14892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10164, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2995}, {'shards': '0,2', 'objects': 1322}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'13478", 'reported_seq': 18717, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:29.206994+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:29.206994+0000', 'last_peered': '2024-09-05T13:18:29.206994+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:29.206994+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29286400, 'num_objects': 7150, 'num_object_clones': 0, 'num_object_copies': 14300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 614, 'num_objects_misplaced': 1274, 'num_objects_unfound': 0, 'num_objects_dirty': 7150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9797, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2550, 'num_bytes_recovered': 10436608, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5262}, {'shards': '0,2', 'objects': 1274}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'13827", 'reported_seq': 13961, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:33.364577+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:33.364577+0000', 'last_peered': '2024-09-05T13:18:33.364577+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:33.364577+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29761536, 'num_objects': 7266, 'num_object_clones': 0, 'num_object_copies': 14532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7266, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11481, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1174}, {'shards': '2', 'objects': 2815}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:34.116 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'13604", 'reported_seq': 13743, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:29.169917+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:29.169917+0000', 'last_peered': '2024-09-05T13:18:29.169917+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:29.169917+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29925376, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9971, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2852}, {'shards': '0,2', 'objects': 1309}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'13551", 'reported_seq': 13675, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:33.364315+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:33.364315+0000', 'last_peered': '2024-09-05T13:18:33.364315+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:33.364315+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28798976, 'num_objects': 7031, 'num_object_clones': 0, 'num_object_copies': 14062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7031, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11060, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1130}, {'shards': '2', 'objects': 2644}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'13576", 'reported_seq': 13717, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:29.146376+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:29.146376+0000', 'last_peered': '2024-09-05T13:18:29.146376+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:29.146376+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29908992, 'num_objects': 7302, 'num_object_clones': 0, 'num_object_copies': 14604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9727, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2883}, {'shards': '0,2', 'objects': 1283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'13729", 'reported_seq': 13917, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:29.171762+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:29.171762+0000', 'last_peered': '2024-09-05T13:18:29.171762+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:29.171762+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30208000, 'num_objects': 7375, 'num_object_clones': 0, 'num_object_copies': 14750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9628, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2931}, {'shards': '0,2', 'objects': 1271}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'13777", 'reported_seq': 13913, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:29.196673+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:29.196673+0000', 'last_peered': '2024-09-05T13:18:29.196673+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:29.196673+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29913088, 'num_objects': 7303, 'num_object_clones': 0, 'num_object_copies': 14606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7303, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9907, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2737}, {'shards': '0,2', 'objects': 1331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:34.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'13796", 'reported_seq': 13937, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:29.239833+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:29.239833+0000', 'last_peered': '2024-09-05T13:18:29.239833+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:29.239833+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30294499, 'num_objects': 7397, 'num_object_clones': 0, 'num_object_copies': 14794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7397, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10464, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2832}, {'shards': '0,2', 'objects': 1371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:34.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:34.813 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:34.814 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45282/232870 objects degraded (19.445%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":441,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":116435,"data_bytes":476901902,"bytes_used":2161156096,"bytes_avail":287749136384,"bytes_total":289910292480,"degraded_objects":45282,"degraded_total":232870,"degraded_ratio":0.19445184008244942,"misplaced_objects":1274,"misplaced_total":232870,"misplaced_ratio":0.0054708635719500154,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":73700,"recovering_keys_per_sec":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":442},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:34.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:34.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:35.407 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:35.410 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:35.431 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-05T13:18:33.650433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":476901902,"num_objects":116435,"num_object_clones":0,"num_object_copies":232870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45282,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":116435,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174756,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6943,"num_bytes_recovered":28418067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110504,"kb_used_data":1092280,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281005016,"statfs":{"total":289910292480,"available":287749136384,"internally_reserved":0,"allocated":1118494720,"data_stored":1117646933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1042598918},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,6,8,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21753856,"num_objects":-5311,"num_object_clones":0,"num_object_copies":-10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-5311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5311,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":884736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004480"},"pg_stats":[{"pgid":"2.f","version":"45'13577","reported_seq":13652,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.215705+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:29.215705+0000","last_peered":"2024-09-05T13:18:29.215705+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:29.215705+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13577,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2974},{"shards":"0,2","objects":1329}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13658","reported_seq":13763,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.320747+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:33.320747+0000","last_peered":"2024-09-05T13:18:33.320747+0000","last_clean":"2024-09-05T13:18:33.320747+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:33.320747+0000","last_undegraded":"2024-09-05T13:18:33.320747+0000","last_fullsized":"2024-09-05T13:18:33.320747+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11668,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13951","reported_seq":14080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.203915+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:33.203915+0000","last_peered":"2024-09-05T13:18:33.203915+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:33.203915+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671448,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11762,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1141},{"shards":"2","objects":2753}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13732","reported_seq":13802,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.340867+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:33.340867+0000","last_peered":"2024-09-05T13:18:33.340867+0000","last_clean":"2024-09-05T13:18:33.340867+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:33.340867+0000","last_undegraded":"2024-09-05T13:18:33.340867+0000","last_fullsized":"2024-09-05T13:18:33.340867+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13732,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13628","reported_seq":13763,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:29.187760+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:29.187760+0000","last_peered":"2024-09-05T13:18:29.187760+0000","last_clean":"2024-09-05T13:18:29.187760+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:29.187760+0000","last_undegraded":"2024-09-05T13:18:29.187760+0000","last_fullsized":"2024-09-05T13:18:29.187760+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29716480,"num_objects":7255,"num_object_clones":0,"num_object_copies":14510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10168,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13709","reported_seq":13848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.053063+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:29.053063+0000","last_peered":"2024-09-05T13:18:29.053063+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:29.053063+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2995},{"shards":"0,2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13478","reported_seq":18717,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.206994+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:29.206994+0000","last_peered":"2024-09-05T13:18:29.206994+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:29.206994+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29286400,"num_objects":7150,"num_object_clones":0,"num_object_copies":14300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":614,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":7150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9797,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2550,"num_bytes_recovered":10436608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13741","reported_seq":13872,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:29.154856+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:29.154856+0000","last_peered":"2024-09-05T13:18:29.154856+0000","last_clean":"2024-09-05T13:18:29.154856+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:29.154856+0000","last_undegraded":"2024-09-05T13:18:29.154856+0000","last_fullsized":"2024-09-05T13:18:29.154856+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30158848,"num_objects":7363,"num_object_clones":0,"num_object_copies":14726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10351,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13827","reported_seq":13961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.364577+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:33.364577+0000","last_peered":"2024-09-05T13:18:33.364577+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:33.364577+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29761536,"num_objects":7266,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11481,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1174},{"shards":"2","objects":2815}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13604","reported_seq":13743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.169917+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:29.169917+0000","last_peered":"2024-09-05T13:18:29.169917+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:29.169917+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29925376,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9971,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2852},{"shards":"0,2","objects":1309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13702","reported_seq":22604,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.371473+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:33.371473+0000","last_peered":"2024-09-05T13:18:33.371473+0000","last_clean":"2024-09-05T13:18:33.371473+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:33.371473+0000","last_undegraded":"2024-09-05T13:18:33.371473+0000","last_fullsized":"2024-09-05T13:18:33.371473+0000","mapping_epoch":44,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11297,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13551","reported_seq":13675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.364315+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:33.364315+0000","last_peered":"2024-09-05T13:18:33.364315+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:33.364315+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11060,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1130},{"shards":"2","objects":2644}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13576","reported_seq":13717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.146376+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:29.146376+0000","last_peered":"2024-09-05T13:18:29.146376+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:29.146376+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29908992,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2883},{"shards":"0,2","objects":1283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13729","reported_seq":13917,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.171762+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:29.171762+0000","last_peered":"2024-09-05T13:18:29.171762+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:29.171762+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30208000,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9628,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2931},{"shards":"0,2","objects":1271}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13777","reported_seq":13913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.196673+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:29.196673+0000","last_peered":"2024-09-05T13:18:29.196673+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:29.196673+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29913088,"num_objects":7303,"num_object_clones":0,"num_object_copies":14606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9907,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2737},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13796","reported_seq":13937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:29.239833+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:29.239833+0000","last_peered":"2024-09-05T13:18:29.239833+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:29.239833+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294499,"num_objects":7397,"num_object_clones":0,"num_object_copies":14794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2832},{"shards":"0,2","objects":1371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":476901883,"num_objects":116433,"num_object_clones":0,"num_object_copies":232866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45282,"num_objects_misplaced":1274,"num_objects_unfound":0,"num_objects_dirty":116433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174754,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6942,"num_bytes_recovered":28418048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1115996160,"data_stored":1115912143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680844,"kb_used_data":374268,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93690996,"statfs":{"total":96636764160,"available":95939579904,"internally_reserved":0,"allocated":383250432,"data_stored":382990875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824316,"kb_used_data":423404,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93547524,"statfs":{"total":96636764160,"available":95792664576,"internally_reserved":0,"allocated":433565696,"data_stored":433306134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605344,"kb_used_data":294608,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93766496,"statfs":{"total":96636764160,"available":96016891904,"internally_reserved":0,"allocated":301678592,"data_stored":301349924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432615424,"data_stored":432611811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301252608,"data_stored":301179889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":382128128,"data_stored":382120443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:38.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:39.001 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:39.001 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:39.023 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-05T13:18:37.651795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471335438,"num_objects":115076,"num_object_clones":0,"num_object_copies":230152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1144,"num_objects_unfound":0,"num_objects_dirty":115076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176115,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7072,"num_bytes_recovered":28946451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082444,"kb_used_data":1085020,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281033076,"statfs":{"total":289910292480,"available":287777869824,"internally_reserved":0,"allocated":1111060480,"data_stored":1110212693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1021299718},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,3,7,8,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18280448,"num_objects":-4463,"num_object_clones":0,"num_object_copies":-8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":-4463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004597"},"pg_stats":[{"pgid":"2.f","version":"45'13701","reported_seq":13775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.233154+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:34.233154+0000","last_peered":"2024-09-05T13:18:34.233154+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:34.233154+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782016,"num_objects":7271,"num_object_clones":0,"num_object_copies":14542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13701,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2850},{"shards":"0,2","objects":1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13658","reported_seq":13763,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.320747+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:33.320747+0000","last_peered":"2024-09-05T13:18:33.320747+0000","last_clean":"2024-09-05T13:18:33.320747+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:33.320747+0000","last_undegraded":"2024-09-05T13:18:33.320747+0000","last_fullsized":"2024-09-05T13:18:33.320747+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11668,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13951","reported_seq":14080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.203915+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:33.203915+0000","last_peered":"2024-09-05T13:18:33.203915+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:33.203915+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671448,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11762,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1141},{"shards":"2","objects":2753}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13732","reported_seq":13802,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.340867+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:33.340867+0000","last_peered":"2024-09-05T13:18:33.340867+0000","last_clean":"2024-09-05T13:18:33.340867+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:33.340867+0000","last_undegraded":"2024-09-05T13:18:33.340867+0000","last_fullsized":"2024-09-05T13:18:33.340867+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13732,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13749","reported_seq":13884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:34.230574+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:34.230574+0000","last_peered":"2024-09-05T13:18:34.230574+0000","last_clean":"2024-09-05T13:18:34.230574+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:34.230574+0000","last_undegraded":"2024-09-05T13:18:34.230574+0000","last_fullsized":"2024-09-05T13:18:34.230574+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29220864,"num_objects":7134,"num_object_clones":0,"num_object_copies":14268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10289,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13849","reported_seq":13988,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.158713+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:34.158713+0000","last_peered":"2024-09-05T13:18:34.158713+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:34.158713+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29925376,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10304,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2855},{"shards":"0,2","objects":1182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13594","reported_seq":19092,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.254346+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:34.254346+0000","last_peered":"2024-09-05T13:18:34.254346+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:34.254346+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28811264,"num_objects":7034,"num_object_clones":0,"num_object_copies":14068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":615,"num_objects_misplaced":1144,"num_objects_unfound":0,"num_objects_dirty":7034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9913,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2679,"num_bytes_recovered":10964992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5275},{"shards":"0,2","objects":1144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13871","reported_seq":14004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:34.115355+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:34.115355+0000","last_peered":"2024-09-05T13:18:34.115355+0000","last_clean":"2024-09-05T13:18:34.115355+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:34.115355+0000","last_undegraded":"2024-09-05T13:18:34.115355+0000","last_fullsized":"2024-09-05T13:18:34.115355+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29626368,"num_objects":7233,"num_object_clones":0,"num_object_copies":14466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10481,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13827","reported_seq":13961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.364577+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:33.364577+0000","last_peered":"2024-09-05T13:18:33.364577+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:33.364577+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29761536,"num_objects":7266,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11481,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1174},{"shards":"2","objects":2815}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13741","reported_seq":13880,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.266213+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:34.266213+0000","last_peered":"2024-09-05T13:18:34.266213+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:34.266213+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364224,"num_objects":7169,"num_object_clones":0,"num_object_copies":14338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10108,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":1172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13702","reported_seq":22604,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.371473+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:33.371473+0000","last_peered":"2024-09-05T13:18:33.371473+0000","last_clean":"2024-09-05T13:18:33.371473+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:33.371473+0000","last_undegraded":"2024-09-05T13:18:33.371473+0000","last_fullsized":"2024-09-05T13:18:33.371473+0000","mapping_epoch":44,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11297,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13551","reported_seq":13675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.364315+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:33.364315+0000","last_peered":"2024-09-05T13:18:33.364315+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:33.364315+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11060,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1130},{"shards":"2","objects":2644}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13721","reported_seq":13859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.204074+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:34.204074+0000","last_peered":"2024-09-05T13:18:34.204074+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:34.204074+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2738},{"shards":"0,2","objects":1138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13866","reported_seq":14054,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.271693+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:34.271693+0000","last_peered":"2024-09-05T13:18:34.271693+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:34.271693+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9765,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2794},{"shards":"0,2","objects":1134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13940","reported_seq":14080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.235561+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:34.235561+0000","last_peered":"2024-09-05T13:18:34.235561+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:34.235561+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29245440,"num_objects":7140,"num_object_clones":0,"num_object_copies":14280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10070,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2574},{"shards":"0,2","objects":1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13942","reported_seq":14083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.251155+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:34.251155+0000","last_peered":"2024-09-05T13:18:34.251155+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:34.251155+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29696483,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2686},{"shards":"0,2","objects":1225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471335419,"num_objects":115074,"num_object_clones":0,"num_object_copies":230148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1144,"num_objects_unfound":0,"num_objects_dirty":115074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176113,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7071,"num_bytes_recovered":28946432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1108660224,"data_stored":1108576207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680844,"kb_used_data":374268,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93690996,"statfs":{"total":96636764160,"available":95939579904,"internally_reserved":0,"allocated":383250432,"data_stored":382990875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":805120,"kb_used_data":416688,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93566720,"statfs":{"total":96636764160,"available":95812321280,"internally_reserved":0,"allocated":426688512,"data_stored":426428950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596480,"kb_used_data":294064,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93775360,"statfs":{"total":96636764160,"available":96025968640,"internally_reserved":0,"allocated":301121536,"data_stored":300792868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425869312,"data_stored":425865699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300662784,"data_stored":300590065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":382128128,"data_stored":382120443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:39.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:39.655 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:39.658 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:39.676 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-05T13:18:37.651795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471335438,"num_objects":115076,"num_object_clones":0,"num_object_copies":230152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1144,"num_objects_unfound":0,"num_objects_dirty":115076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176115,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7072,"num_bytes_recovered":28946451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082444,"kb_used_data":1085020,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281033076,"statfs":{"total":289910292480,"available":287777869824,"internally_reserved":0,"allocated":1111060480,"data_stored":1110212693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1021299718},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,3,7,8,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18280448,"num_objects":-4463,"num_object_clones":0,"num_object_copies":-8926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":-4463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4463,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004597"},"pg_stats":[{"pgid":"2.f","version":"45'13701","reported_seq":13775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.233154+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:34.233154+0000","last_peered":"2024-09-05T13:18:34.233154+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:34.233154+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782016,"num_objects":7271,"num_object_clones":0,"num_object_copies":14542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13701,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2850},{"shards":"0,2","objects":1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13658","reported_seq":13763,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.320747+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:33.320747+0000","last_peered":"2024-09-05T13:18:33.320747+0000","last_clean":"2024-09-05T13:18:33.320747+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:33.320747+0000","last_undegraded":"2024-09-05T13:18:33.320747+0000","last_fullsized":"2024-09-05T13:18:33.320747+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11668,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13951","reported_seq":14080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.203915+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:33.203915+0000","last_peered":"2024-09-05T13:18:33.203915+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:33.203915+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671448,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11762,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1141},{"shards":"2","objects":2753}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13732","reported_seq":13802,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.340867+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:33.340867+0000","last_peered":"2024-09-05T13:18:33.340867+0000","last_clean":"2024-09-05T13:18:33.340867+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:33.340867+0000","last_undegraded":"2024-09-05T13:18:33.340867+0000","last_fullsized":"2024-09-05T13:18:33.340867+0000","mapping_epoch":35,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29876224,"num_objects":7294,"num_object_clones":0,"num_object_copies":14588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13732,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13749","reported_seq":13884,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:34.230574+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:34.230574+0000","last_peered":"2024-09-05T13:18:34.230574+0000","last_clean":"2024-09-05T13:18:34.230574+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:34.230574+0000","last_undegraded":"2024-09-05T13:18:34.230574+0000","last_fullsized":"2024-09-05T13:18:34.230574+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29220864,"num_objects":7134,"num_object_clones":0,"num_object_copies":14268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10289,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13849","reported_seq":13988,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.158713+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:34.158713+0000","last_peered":"2024-09-05T13:18:34.158713+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:34.158713+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29925376,"num_objects":7306,"num_object_clones":0,"num_object_copies":14612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10304,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2855},{"shards":"0,2","objects":1182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13594","reported_seq":19092,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.254346+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:34.254346+0000","last_peered":"2024-09-05T13:18:34.254346+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:34.254346+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28811264,"num_objects":7034,"num_object_clones":0,"num_object_copies":14068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":615,"num_objects_misplaced":1144,"num_objects_unfound":0,"num_objects_dirty":7034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9913,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2679,"num_bytes_recovered":10964992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5275},{"shards":"0,2","objects":1144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13871","reported_seq":14004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:34.115355+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:34.115355+0000","last_peered":"2024-09-05T13:18:34.115355+0000","last_clean":"2024-09-05T13:18:34.115355+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:34.115355+0000","last_undegraded":"2024-09-05T13:18:34.115355+0000","last_fullsized":"2024-09-05T13:18:34.115355+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29626368,"num_objects":7233,"num_object_clones":0,"num_object_copies":14466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10481,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13827","reported_seq":13961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.364577+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:33.364577+0000","last_peered":"2024-09-05T13:18:33.364577+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:33.364577+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29761536,"num_objects":7266,"num_object_clones":0,"num_object_copies":14532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11481,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1174},{"shards":"2","objects":2815}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13741","reported_seq":13880,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.266213+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:34.266213+0000","last_peered":"2024-09-05T13:18:34.266213+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:34.266213+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364224,"num_objects":7169,"num_object_clones":0,"num_object_copies":14338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10108,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":1172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13702","reported_seq":22604,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:33.371473+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:33.371473+0000","last_peered":"2024-09-05T13:18:33.371473+0000","last_clean":"2024-09-05T13:18:33.371473+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:33.371473+0000","last_undegraded":"2024-09-05T13:18:33.371473+0000","last_fullsized":"2024-09-05T13:18:33.371473+0000","mapping_epoch":44,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11297,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13551","reported_seq":13675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:33.364315+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:33.364315+0000","last_peered":"2024-09-05T13:18:33.364315+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:33.364315+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11060,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1130},{"shards":"2","objects":2644}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13721","reported_seq":13859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.204074+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:34.204074+0000","last_peered":"2024-09-05T13:18:34.204074+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:34.204074+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29315072,"num_objects":7157,"num_object_clones":0,"num_object_copies":14314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2738},{"shards":"0,2","objects":1138}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13866","reported_seq":14054,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.271693+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:34.271693+0000","last_peered":"2024-09-05T13:18:34.271693+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:34.271693+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29646848,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9765,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2794},{"shards":"0,2","objects":1134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13940","reported_seq":14080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.235561+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:34.235561+0000","last_peered":"2024-09-05T13:18:34.235561+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:34.235561+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29245440,"num_objects":7140,"num_object_clones":0,"num_object_copies":14280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10070,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2574},{"shards":"0,2","objects":1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13942","reported_seq":14083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:34.251155+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:34.251155+0000","last_peered":"2024-09-05T13:18:34.251155+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:34.251155+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29696483,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10610,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2686},{"shards":"0,2","objects":1225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471335419,"num_objects":115074,"num_object_clones":0,"num_object_copies":230148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1144,"num_objects_unfound":0,"num_objects_dirty":115074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176113,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7071,"num_bytes_recovered":28946432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1108660224,"data_stored":1108576207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680844,"kb_used_data":374268,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93690996,"statfs":{"total":96636764160,"available":95939579904,"internally_reserved":0,"allocated":383250432,"data_stored":382990875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":805120,"kb_used_data":416688,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93566720,"statfs":{"total":96636764160,"available":95812321280,"internally_reserved":0,"allocated":426688512,"data_stored":426428950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596480,"kb_used_data":294064,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93775360,"statfs":{"total":96636764160,"available":96025968640,"internally_reserved":0,"allocated":301121536,"data_stored":300792868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425869312,"data_stored":425865699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300662784,"data_stored":300590065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":382128128,"data_stored":382120443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:39.678 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:39.678 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13701", 'reported_seq': 13775, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:34.233154+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:34.233154+0000', 'last_peered': '2024-09-05T13:18:34.233154+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:34.233154+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29782016, 'num_objects': 7271, 'num_object_clones': 0, 'num_object_copies': 14542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13701, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2850}, {'shards': '0,2', 'objects': 1205}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:39.678 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:39.678 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'13951", 'reported_seq': 14080, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:33.203915+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:33.203915+0000', 'last_peered': '2024-09-05T13:18:33.203915+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:33.203915+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29671448, 'num_objects': 7245, 'num_object_clones': 0, 'num_object_copies': 14490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7245, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11762, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1141}, {'shards': '2', 'objects': 2753}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:39.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:39.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'13849", 'reported_seq': 13988, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:34.158713+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:34.158713+0000', 'last_peered': '2024-09-05T13:18:34.158713+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:34.158713+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29925376, 'num_objects': 7306, 'num_object_clones': 0, 'num_object_copies': 14612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10304, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2855}, {'shards': '0,2', 'objects': 1182}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:39.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:39.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'13594", 'reported_seq': 19092, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:34.254346+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:34.254346+0000', 'last_peered': '2024-09-05T13:18:34.254346+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:34.254346+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28811264, 'num_objects': 7034, 'num_object_clones': 0, 'num_object_copies': 14068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 615, 'num_objects_misplaced': 1144, 'num_objects_unfound': 0, 'num_objects_dirty': 7034, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9913, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2679, 'num_bytes_recovered': 10964992, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5275}, {'shards': '0,2', 'objects': 1144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:39.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:39.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'13827", 'reported_seq': 13961, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:33.364577+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:33.364577+0000', 'last_peered': '2024-09-05T13:18:33.364577+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:33.364577+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29761536, 'num_objects': 7266, 'num_object_clones': 0, 'num_object_copies': 14532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7266, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11481, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1174}, {'shards': '2', 'objects': 2815}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:39.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:39.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'13741", 'reported_seq': 13880, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:34.266213+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:34.266213+0000', 'last_peered': '2024-09-05T13:18:34.266213+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:34.266213+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29364224, 'num_objects': 7169, 'num_object_clones': 0, 'num_object_copies': 14338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10108, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2715}, {'shards': '0,2', 'objects': 1172}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:39.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:39.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'13551", 'reported_seq': 13675, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:33.364315+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:33.364315+0000', 'last_peered': '2024-09-05T13:18:33.364315+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:33.364315+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28798976, 'num_objects': 7031, 'num_object_clones': 0, 'num_object_copies': 14062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7031, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11060, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1130}, {'shards': '2', 'objects': 2644}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:39.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:39.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'13721", 'reported_seq': 13859, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:34.204074+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:34.204074+0000', 'last_peered': '2024-09-05T13:18:34.204074+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:34.204074+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29315072, 'num_objects': 7157, 'num_object_clones': 0, 'num_object_copies': 14314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9872, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2738}, {'shards': '0,2', 'objects': 1138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:39.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:39.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'13866", 'reported_seq': 14054, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:34.271693+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:34.271693+0000', 'last_peered': '2024-09-05T13:18:34.271693+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:34.271693+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29646848, 'num_objects': 7238, 'num_object_clones': 0, 'num_object_copies': 14476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9765, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2794}, {'shards': '0,2', 'objects': 1134}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:39.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:39.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'13940", 'reported_seq': 14080, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:34.235561+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:34.235561+0000', 'last_peered': '2024-09-05T13:18:34.235561+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:34.235561+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29245440, 'num_objects': 7140, 'num_object_clones': 0, 'num_object_copies': 14280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7140, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10070, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2574}, {'shards': '0,2', 'objects': 1168}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:39.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:39.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'13942", 'reported_seq': 14083, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:34.251155+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:34.251155+0000', 'last_peered': '2024-09-05T13:18:34.251155+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:34.251155+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29696483, 'num_objects': 7251, 'num_object_clones': 0, 'num_object_copies': 14502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7251, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10610, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2686}, {'shards': '0,2', 'objects': 1225}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:39.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:40.382 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:40.382 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45283/225648 objects degraded (20.068%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":447,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112824,"data_bytes":462111246,"bytes_used":2134011904,"bytes_avail":287776280576,"bytes_total":289910292480,"degraded_objects":45283,"degraded_total":225648,"degraded_ratio":0.20067981989647593,"misplaced_objects":1022,"misplaced_total":225648,"misplaced_ratio":0.0045291781890377933,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":123155,"recovering_keys_per_sec":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":559},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:40.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:40.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:40.994 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:40.994 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:41.016 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-05T13:18:39.654208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":462111246,"num_objects":112824,"num_object_clones":0,"num_object_copies":225648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":112824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178367,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7194,"num_bytes_recovered":29446163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160650,"ondisk_log_size":160650,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2083996,"kb_used_data":1074092,"kb_used_omap":12,"kb_used_meta":1009843,"kb_avail":281031524,"statfs":{"total":289910292480,"available":287776280576,"internally_reserved":0,"allocated":1099870208,"data_stored":1099022421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1034079238},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,2,2,5,9,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27504640,"num_objects":-6715,"num_object_clones":0,"num_object_copies":-13430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":-388,"num_objects_unfound":0,"num_objects_dirty":-6715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006422"},"pg_stats":[{"pgid":"2.f","version":"45'13846","reported_seq":13921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.298571+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:39.298571+0000","last_peered":"2024-09-05T13:18:39.298571+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:39.298571+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13846,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2705},{"shards":"0,2","objects":1060}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13801","reported_seq":13906,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:38.280152+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:38.280152+0000","last_peered":"2024-09-05T13:18:38.280152+0000","last_clean":"2024-09-05T13:18:38.280152+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:38.280152+0000","last_undegraded":"2024-09-05T13:18:38.280152+0000","last_fullsized":"2024-09-05T13:18:38.280152+0000","mapping_epoch":35,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28479488,"num_objects":6953,"num_object_clones":0,"num_object_copies":13906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11811,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14101","reported_seq":14228,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:38.369952+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:38.369952+0000","last_peered":"2024-09-05T13:18:38.369952+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:38.369952+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057048,"num_objects":7095,"num_object_clones":0,"num_object_copies":14190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11912,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":991},{"shards":"2","objects":2603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13877","reported_seq":13947,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:38.350618+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:38.350618+0000","last_peered":"2024-09-05T13:18:38.350618+0000","last_clean":"2024-09-05T13:18:38.350618+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:38.350618+0000","last_undegraded":"2024-09-05T13:18:38.350618+0000","last_fullsized":"2024-09-05T13:18:38.350618+0000","mapping_epoch":35,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29282304,"num_objects":7149,"num_object_clones":0,"num_object_copies":14298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13877,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13881","reported_seq":14016,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:39.107929+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:39.107929+0000","last_peered":"2024-09-05T13:18:39.107929+0000","last_clean":"2024-09-05T13:18:39.107929+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:39.107929+0000","last_undegraded":"2024-09-05T13:18:39.107929+0000","last_fullsized":"2024-09-05T13:18:39.107929+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28680192,"num_objects":7002,"num_object_clones":0,"num_object_copies":14004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10421,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14012","reported_seq":14151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.257866+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:39.257866+0000","last_peered":"2024-09-05T13:18:39.257866+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:39.257866+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29257728,"num_objects":7143,"num_object_clones":0,"num_object_copies":14286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10467,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2692},{"shards":"0,2","objects":1019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13719","reported_seq":19462,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.298113+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:39.298113+0000","last_peered":"2024-09-05T13:18:39.298113+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:39.298113+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":615,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10038,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2801,"num_bytes_recovered":11464704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":1022}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14010","reported_seq":14143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:39.165586+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:39.165586+0000","last_peered":"2024-09-05T13:18:39.165586+0000","last_clean":"2024-09-05T13:18:39.165586+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:39.165586+0000","last_undegraded":"2024-09-05T13:18:39.165586+0000","last_fullsized":"2024-09-05T13:18:39.165586+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10620,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13962","reported_seq":14097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:38.338497+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:38.338497+0000","last_peered":"2024-09-05T13:18:38.338497+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:38.338497+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29208576,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11616,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1039},{"shards":"2","objects":2680}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13878","reported_seq":14017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.265133+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:39.265133+0000","last_peered":"2024-09-05T13:18:39.265133+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:39.265133+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28803072,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10245,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":1035}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13842","reported_seq":22744,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:38.326027+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:38.326027+0000","last_peered":"2024-09-05T13:18:38.326027+0000","last_clean":"2024-09-05T13:18:38.326027+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:38.326027+0000","last_undegraded":"2024-09-05T13:18:38.326027+0000","last_fullsized":"2024-09-05T13:18:38.326027+0000","mapping_epoch":44,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11437,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13684","reported_seq":13808,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:38.285566+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:38.285566+0000","last_peered":"2024-09-05T13:18:38.285566+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:38.285566+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28254208,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11193,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":997},{"shards":"2","objects":2511}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13856","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.268023+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:39.268023+0000","last_peered":"2024-09-05T13:18:39.268023+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:39.268023+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2603},{"shards":"0,2","objects":1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14001","reported_seq":14189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.144720+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:39.144720+0000","last_peered":"2024-09-05T13:18:39.144720+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:39.144720+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29093888,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9900,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2659},{"shards":"0,2","objects":999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14073","reported_seq":14211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.307301+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:39.307301+0000","last_peered":"2024-09-05T13:18:39.307301+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:39.307301+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10203,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2441},{"shards":"0,2","objects":1035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14104","reported_seq":14243,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.260328+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:39.260328+0000","last_peered":"2024-09-05T13:18:39.260328+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:39.260328+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29032931,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10772,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2524},{"shards":"0,2","objects":1063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":462111227,"num_objects":112822,"num_object_clones":0,"num_object_copies":225644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":112822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178365,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7193,"num_bytes_recovered":29446144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1097285632,"data_stored":1097201615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160647,"ondisk_log_size":160647,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680676,"kb_used_data":369940,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93691164,"statfs":{"total":96636764160,"available":95939751936,"internally_reserved":0,"allocated":378818560,"data_stored":378559003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":806840,"kb_used_data":410088,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93565000,"statfs":{"total":96636764160,"available":95810560000,"internally_reserved":0,"allocated":419930112,"data_stored":419670550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,1,2,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596480,"kb_used_data":294064,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93775360,"statfs":{"total":96636764160,"available":96025968640,"internally_reserved":0,"allocated":301121536,"data_stored":300792868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419012608,"data_stored":419008995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300662784,"data_stored":300590065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":377610240,"data_stored":377602555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:44.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:44.559 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:44.559 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:44.583 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-05T13:18:43.655619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":458875406,"num_objects":112034,"num_object_clones":0,"num_object_copies":224068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":112034,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179157,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7194,"num_bytes_recovered":29446163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160540,"ondisk_log_size":160540,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2088984,"kb_used_data":1069352,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":281026536,"statfs":{"total":289910292480,"available":287771172864,"internally_reserved":0,"allocated":1095016448,"data_stored":1094168661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1044040710},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,4,8,7,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21467136,"num_objects":-5241,"num_object_clones":0,"num_object_copies":-10482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":-5241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5241,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006000"},"pg_stats":[{"pgid":"2.f","version":"45'13846","reported_seq":13921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.298571+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:39.298571+0000","last_peered":"2024-09-05T13:18:39.298571+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:39.298571+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13846,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2705},{"shards":"0,2","objects":1060}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13932","reported_seq":14037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.328971+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:43.328971+0000","last_peered":"2024-09-05T13:18:43.328971+0000","last_clean":"2024-09-05T13:18:43.328971+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:43.328971+0000","last_undegraded":"2024-09-05T13:18:43.328971+0000","last_fullsized":"2024-09-05T13:18:43.328971+0000","mapping_epoch":35,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27942912,"num_objects":6822,"num_object_clones":0,"num_object_copies":13644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11942,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14232","reported_seq":14361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.349649+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:43.349649+0000","last_peered":"2024-09-05T13:18:43.349649+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:43.349649+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520472,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12043,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":860},{"shards":"2","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14006","reported_seq":14076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.271361+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:43.271361+0000","last_peered":"2024-09-05T13:18:43.271361+0000","last_clean":"2024-09-05T13:18:43.271361+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:43.271361+0000","last_undegraded":"2024-09-05T13:18:43.271361+0000","last_fullsized":"2024-09-05T13:18:43.271361+0000","mapping_epoch":35,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28753920,"num_objects":7020,"num_object_clones":0,"num_object_copies":14040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14006,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13881","reported_seq":14016,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:39.107929+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:39.107929+0000","last_peered":"2024-09-05T13:18:39.107929+0000","last_clean":"2024-09-05T13:18:39.107929+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:39.107929+0000","last_undegraded":"2024-09-05T13:18:39.107929+0000","last_fullsized":"2024-09-05T13:18:39.107929+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28680192,"num_objects":7002,"num_object_clones":0,"num_object_copies":14004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10421,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14012","reported_seq":14151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.257866+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:39.257866+0000","last_peered":"2024-09-05T13:18:39.257866+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:39.257866+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29257728,"num_objects":7143,"num_object_clones":0,"num_object_copies":14286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10467,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2692},{"shards":"0,2","objects":1019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13719","reported_seq":19462,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.298113+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:39.298113+0000","last_peered":"2024-09-05T13:18:39.298113+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:39.298113+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":615,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10038,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2801,"num_bytes_recovered":11464704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":1022}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14010","reported_seq":14143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:39.165586+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:39.165586+0000","last_peered":"2024-09-05T13:18:39.165586+0000","last_clean":"2024-09-05T13:18:39.165586+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:39.165586+0000","last_undegraded":"2024-09-05T13:18:39.165586+0000","last_fullsized":"2024-09-05T13:18:39.165586+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10620,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14106","reported_seq":14239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.388851+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:43.388851+0000","last_peered":"2024-09-05T13:18:43.388851+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:43.388851+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618752,"num_objects":6987,"num_object_clones":0,"num_object_copies":13974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11760,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":895},{"shards":"2","objects":2536}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13878","reported_seq":14017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.265133+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:39.265133+0000","last_peered":"2024-09-05T13:18:39.265133+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:39.265133+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28803072,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10245,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":1035}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13971","reported_seq":22873,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.389198+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:43.389198+0000","last_peered":"2024-09-05T13:18:43.389198+0000","last_clean":"2024-09-05T13:18:43.389198+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:43.389198+0000","last_undegraded":"2024-09-05T13:18:43.389198+0000","last_fullsized":"2024-09-05T13:18:43.389198+0000","mapping_epoch":44,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11566,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13810","reported_seq":13934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.403714+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:43.403714+0000","last_peered":"2024-09-05T13:18:43.403714+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:43.403714+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11319,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":871},{"shards":"2","objects":2385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13856","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.268023+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:39.268023+0000","last_peered":"2024-09-05T13:18:39.268023+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:39.268023+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2603},{"shards":"0,2","objects":1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14001","reported_seq":14189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.144720+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:39.144720+0000","last_peered":"2024-09-05T13:18:39.144720+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:39.144720+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29093888,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9900,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2659},{"shards":"0,2","objects":999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14073","reported_seq":14211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.307301+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:39.307301+0000","last_peered":"2024-09-05T13:18:39.307301+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:39.307301+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10203,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2441},{"shards":"0,2","objects":1035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14104","reported_seq":14243,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.260328+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:39.260328+0000","last_peered":"2024-09-05T13:18:39.260328+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:39.260328+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29032931,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10772,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2524},{"shards":"0,2","objects":1063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":458875387,"num_objects":112032,"num_object_clones":0,"num_object_copies":224064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":112032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179155,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7193,"num_bytes_recovered":29446144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092366336,"data_stored":1092282319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160537,"ondisk_log_size":160537,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664024,"kb_used_data":365768,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93707816,"statfs":{"total":96636764160,"available":95956803584,"internally_reserved":0,"allocated":374546432,"data_stored":374286875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":806840,"kb_used_data":410088,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93565000,"statfs":{"total":96636764160,"available":95810560000,"internally_reserved":0,"allocated":419930112,"data_stored":419670550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,1,2,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618120,"kb_used_data":293496,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93753720,"statfs":{"total":96636764160,"available":96003809280,"internally_reserved":0,"allocated":300539904,"data_stored":300211236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419012608,"data_stored":419008995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300077056,"data_stored":300004337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":373276672,"data_stored":373268987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:44.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:45.147 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:45.147 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:45.168 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-05T13:18:43.655619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":458875406,"num_objects":112034,"num_object_clones":0,"num_object_copies":224068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":112034,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179157,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7194,"num_bytes_recovered":29446163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160540,"ondisk_log_size":160540,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2088984,"kb_used_data":1069352,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":281026536,"statfs":{"total":289910292480,"available":287771172864,"internally_reserved":0,"allocated":1095016448,"data_stored":1094168661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1044040710},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,4,8,7,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21467136,"num_objects":-5241,"num_object_clones":0,"num_object_copies":-10482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":-5241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5241,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006000"},"pg_stats":[{"pgid":"2.f","version":"45'13846","reported_seq":13921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.298571+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:39.298571+0000","last_peered":"2024-09-05T13:18:39.298571+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:39.298571+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29188096,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13846,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2705},{"shards":"0,2","objects":1060}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13932","reported_seq":14037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.328971+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:43.328971+0000","last_peered":"2024-09-05T13:18:43.328971+0000","last_clean":"2024-09-05T13:18:43.328971+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:43.328971+0000","last_undegraded":"2024-09-05T13:18:43.328971+0000","last_fullsized":"2024-09-05T13:18:43.328971+0000","mapping_epoch":35,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27942912,"num_objects":6822,"num_object_clones":0,"num_object_copies":13644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11942,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14232","reported_seq":14361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.349649+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:43.349649+0000","last_peered":"2024-09-05T13:18:43.349649+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:43.349649+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520472,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12043,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":860},{"shards":"2","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14006","reported_seq":14076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.271361+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:43.271361+0000","last_peered":"2024-09-05T13:18:43.271361+0000","last_clean":"2024-09-05T13:18:43.271361+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:43.271361+0000","last_undegraded":"2024-09-05T13:18:43.271361+0000","last_fullsized":"2024-09-05T13:18:43.271361+0000","mapping_epoch":35,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28753920,"num_objects":7020,"num_object_clones":0,"num_object_copies":14040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14006,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13881","reported_seq":14016,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:39.107929+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:39.107929+0000","last_peered":"2024-09-05T13:18:39.107929+0000","last_clean":"2024-09-05T13:18:39.107929+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:39.107929+0000","last_undegraded":"2024-09-05T13:18:39.107929+0000","last_fullsized":"2024-09-05T13:18:39.107929+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28680192,"num_objects":7002,"num_object_clones":0,"num_object_copies":14004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10421,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14012","reported_seq":14151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.257866+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:39.257866+0000","last_peered":"2024-09-05T13:18:39.257866+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:39.257866+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29257728,"num_objects":7143,"num_object_clones":0,"num_object_copies":14286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10467,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2692},{"shards":"0,2","objects":1019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13719","reported_seq":19462,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.298113+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:39.298113+0000","last_peered":"2024-09-05T13:18:39.298113+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:39.298113+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":615,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10038,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2801,"num_bytes_recovered":11464704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":1022}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14010","reported_seq":14143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:39.165586+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:39.165586+0000","last_peered":"2024-09-05T13:18:39.165586+0000","last_clean":"2024-09-05T13:18:39.165586+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:39.165586+0000","last_undegraded":"2024-09-05T13:18:39.165586+0000","last_fullsized":"2024-09-05T13:18:39.165586+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10620,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14106","reported_seq":14239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.388851+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:43.388851+0000","last_peered":"2024-09-05T13:18:43.388851+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:43.388851+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618752,"num_objects":6987,"num_object_clones":0,"num_object_copies":13974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11760,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":895},{"shards":"2","objects":2536}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13878","reported_seq":14017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.265133+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:39.265133+0000","last_peered":"2024-09-05T13:18:39.265133+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:39.265133+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28803072,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10245,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":1035}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13971","reported_seq":22873,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.389198+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:43.389198+0000","last_peered":"2024-09-05T13:18:43.389198+0000","last_clean":"2024-09-05T13:18:43.389198+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:43.389198+0000","last_undegraded":"2024-09-05T13:18:43.389198+0000","last_fullsized":"2024-09-05T13:18:43.389198+0000","mapping_epoch":44,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11566,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13810","reported_seq":13934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.403714+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:43.403714+0000","last_peered":"2024-09-05T13:18:43.403714+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:43.403714+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11319,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":871},{"shards":"2","objects":2385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13856","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.268023+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:39.268023+0000","last_peered":"2024-09-05T13:18:39.268023+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:39.268023+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2603},{"shards":"0,2","objects":1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14001","reported_seq":14189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.144720+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:39.144720+0000","last_peered":"2024-09-05T13:18:39.144720+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:39.144720+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29093888,"num_objects":7103,"num_object_clones":0,"num_object_copies":14206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9900,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2659},{"shards":"0,2","objects":999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14073","reported_seq":14211,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.307301+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:39.307301+0000","last_peered":"2024-09-05T13:18:39.307301+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:39.307301+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28700672,"num_objects":7007,"num_object_clones":0,"num_object_copies":14014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10203,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2441},{"shards":"0,2","objects":1035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14104","reported_seq":14243,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:39.260328+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:39.260328+0000","last_peered":"2024-09-05T13:18:39.260328+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:39.260328+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29032931,"num_objects":7089,"num_object_clones":0,"num_object_copies":14178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10772,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2524},{"shards":"0,2","objects":1063}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":458875387,"num_objects":112032,"num_object_clones":0,"num_object_copies":224064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45283,"num_objects_misplaced":1022,"num_objects_unfound":0,"num_objects_dirty":112032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179155,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7193,"num_bytes_recovered":29446144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092366336,"data_stored":1092282319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160537,"ondisk_log_size":160537,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664024,"kb_used_data":365768,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93707816,"statfs":{"total":96636764160,"available":95956803584,"internally_reserved":0,"allocated":374546432,"data_stored":374286875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":806840,"kb_used_data":410088,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93565000,"statfs":{"total":96636764160,"available":95810560000,"internally_reserved":0,"allocated":419930112,"data_stored":419670550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,1,2,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618120,"kb_used_data":293496,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93753720,"statfs":{"total":96636764160,"available":96003809280,"internally_reserved":0,"allocated":300539904,"data_stored":300211236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419012608,"data_stored":419008995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300077056,"data_stored":300004337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":373276672,"data_stored":373268987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:45.169 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:45.169 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'13846", 'reported_seq': 13921, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:39.298571+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:39.298571+0000', 'last_peered': '2024-09-05T13:18:39.298571+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:39.298571+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29188096, 'num_objects': 7126, 'num_object_clones': 0, 'num_object_copies': 14252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13846, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2705}, {'shards': '0,2', 'objects': 1060}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'14232", 'reported_seq': 14361, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:43.349649+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:43.349649+0000', 'last_peered': '2024-09-05T13:18:43.349649+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:43.349649+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28520472, 'num_objects': 6964, 'num_object_clones': 0, 'num_object_copies': 13928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6964, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12043, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 860}, {'shards': '2', 'objects': 2472}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'14012", 'reported_seq': 14151, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:39.257866+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:39.257866+0000', 'last_peered': '2024-09-05T13:18:39.257866+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:39.257866+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29257728, 'num_objects': 7143, 'num_object_clones': 0, 'num_object_copies': 14286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7143, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10467, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2692}, {'shards': '0,2', 'objects': 1019}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'13719", 'reported_seq': 19462, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:39.298113+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:39.298113+0000', 'last_peered': '2024-09-05T13:18:39.298113+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:39.298113+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28299264, 'num_objects': 6909, 'num_object_clones': 0, 'num_object_copies': 13818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 615, 'num_objects_misplaced': 1022, 'num_objects_unfound': 0, 'num_objects_dirty': 6909, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10038, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2801, 'num_bytes_recovered': 11464704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5272}, {'shards': '0,2', 'objects': 1022}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'14106", 'reported_seq': 14239, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:43.388851+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:43.388851+0000', 'last_peered': '2024-09-05T13:18:43.388851+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:43.388851+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28618752, 'num_objects': 6987, 'num_object_clones': 0, 'num_object_copies': 13974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6987, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11760, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 895}, {'shards': '2', 'objects': 2536}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:45.170 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'13878", 'reported_seq': 14017, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:39.265133+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:39.265133+0000', 'last_peered': '2024-09-05T13:18:39.265133+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:39.265133+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28803072, 'num_objects': 7032, 'num_object_clones': 0, 'num_object_copies': 14064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10245, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2578}, {'shards': '0,2', 'objects': 1035}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:45.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:45.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'13810", 'reported_seq': 13934, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:43.403714+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:43.403714+0000', 'last_peered': '2024-09-05T13:18:43.403714+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:43.403714+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27738112, 'num_objects': 6772, 'num_object_clones': 0, 'num_object_copies': 13544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11319, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 871}, {'shards': '2', 'objects': 2385}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:45.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:45.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'13856", 'reported_seq': 13997, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:39.268023+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:39.268023+0000', 'last_peered': '2024-09-05T13:18:39.268023+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:39.268023+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28762112, 'num_objects': 7022, 'num_object_clones': 0, 'num_object_copies': 14044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10007, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2603}, {'shards': '0,2', 'objects': 1003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:45.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:45.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'14001", 'reported_seq': 14189, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:39.144720+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:39.144720+0000', 'last_peered': '2024-09-05T13:18:39.144720+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:39.144720+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "36'4000", 'ondisk_log_start': "36'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29093888, 'num_objects': 7103, 'num_object_clones': 0, 'num_object_copies': 14206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9900, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2659}, {'shards': '0,2', 'objects': 999}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:45.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:45.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'14073", 'reported_seq': 14211, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:39.307301+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:39.307301+0000', 'last_peered': '2024-09-05T13:18:39.307301+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:39.307301+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "36'4000", 'ondisk_log_start': "36'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28700672, 'num_objects': 7007, 'num_object_clones': 0, 'num_object_copies': 14014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10203, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2441}, {'shards': '0,2', 'objects': 1035}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:45.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:45.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'14104", 'reported_seq': 14243, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:39.260328+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:39.260328+0000', 'last_peered': '2024-09-05T13:18:39.260328+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:39.260328+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29032931, 'num_objects': 7089, 'num_object_clones': 0, 'num_object_copies': 14178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10772, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2524}, {'shards': '0,2', 'objects': 1063}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:45.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:45.888 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:45.888 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45283/224068 objects degraded (20.209%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":452,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112034,"data_bytes":458875406,"bytes_used":2139119616,"bytes_avail":287771172864,"bytes_total":289910292480,"degraded_objects":45283,"degraded_total":224068,"degraded_ratio":0.20209489976257211,"misplaced_objects":1022,"misplaced_total":224068,"misplaced_ratio":0.004561115375689523,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":85631,"recovering_keys_per_sec":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":436},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:45.917 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:45.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:46.482 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:46.507 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-05T13:18:45.656185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452993550,"num_objects":110598,"num_object_clones":0,"num_object_copies":221196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45306,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":110598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180593,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7316,"num_bytes_recovered":29945875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2086120,"kb_used_data":1062328,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281029400,"statfs":{"total":289910292480,"available":287774105600,"internally_reserved":0,"allocated":1087823872,"data_stored":1086976085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1048300550},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,16,9,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23908352,"num_objects":-5837,"num_object_clones":0,"num_object_copies":-11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":-397,"num_objects_unfound":0,"num_objects_dirty":-5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":1527808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005751"},"pg_stats":[{"pgid":"2.f","version":"45'14000","reported_seq":14075,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:44.222598+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:44.222598+0000","last_peered":"2024-09-05T13:18:44.222598+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:44.222598+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28557312,"num_objects":6972,"num_object_clones":0,"num_object_copies":13944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14000,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2551},{"shards":"0,2","objects":906}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13932","reported_seq":14037,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.328971+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:43.328971+0000","last_peered":"2024-09-05T13:18:43.328971+0000","last_clean":"2024-09-05T13:18:43.328971+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:43.328971+0000","last_undegraded":"2024-09-05T13:18:43.328971+0000","last_fullsized":"2024-09-05T13:18:43.328971+0000","mapping_epoch":35,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27942912,"num_objects":6822,"num_object_clones":0,"num_object_copies":13644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11942,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14232","reported_seq":14361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.349649+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:43.349649+0000","last_peered":"2024-09-05T13:18:43.349649+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:43.349649+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520472,"num_objects":6964,"num_object_clones":0,"num_object_copies":13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12043,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":860},{"shards":"2","objects":2472}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14006","reported_seq":14076,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.271361+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:43.271361+0000","last_peered":"2024-09-05T13:18:43.271361+0000","last_clean":"2024-09-05T13:18:43.271361+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:43.271361+0000","last_undegraded":"2024-09-05T13:18:43.271361+0000","last_fullsized":"2024-09-05T13:18:43.271361+0000","mapping_epoch":35,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28753920,"num_objects":7020,"num_object_clones":0,"num_object_copies":14040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14006,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14007","reported_seq":14142,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:44.275400+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:44.275400+0000","last_peered":"2024-09-05T13:18:44.275400+0000","last_clean":"2024-09-05T13:18:44.275400+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:44.275400+0000","last_undegraded":"2024-09-05T13:18:44.275400+0000","last_fullsized":"2024-09-05T13:18:44.275400+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28164096,"num_objects":6876,"num_object_clones":0,"num_object_copies":13752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10547,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14158","reported_seq":14297,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:44.243467+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:44.243467+0000","last_peered":"2024-09-05T13:18:44.243467+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:44.243467+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28659712,"num_objects":6997,"num_object_clones":0,"num_object_copies":13994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10613,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2546},{"shards":"0,2","objects":873}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13875","reported_seq":19863,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:44.333716+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:44.333716+0000","last_peered":"2024-09-05T13:18:44.333716+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:44.333716+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27660288,"num_objects":6753,"num_object_clones":0,"num_object_copies":13506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":638,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":6753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10194,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2923,"num_bytes_recovered":11964416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5238},{"shards":"0,2","objects":877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14160","reported_seq":14293,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:44.252043+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:44.252043+0000","last_peered":"2024-09-05T13:18:44.252043+0000","last_clean":"2024-09-05T13:18:44.252043+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:44.252043+0000","last_undegraded":"2024-09-05T13:18:44.252043+0000","last_fullsized":"2024-09-05T13:18:44.252043+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":6944,"num_object_clones":0,"num_object_copies":13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10770,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14106","reported_seq":14239,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.388851+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:43.388851+0000","last_peered":"2024-09-05T13:18:43.388851+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:43.388851+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618752,"num_objects":6987,"num_object_clones":0,"num_object_copies":13974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11760,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":895},{"shards":"2","objects":2536}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14001","reported_seq":14139,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:44.336462+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:44.336462+0000","last_peered":"2024-09-05T13:18:44.336462+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:44.336462+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28299264,"num_objects":6909,"num_object_clones":0,"num_object_copies":13818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10368,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2455},{"shards":"0,2","objects":912}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13971","reported_seq":22873,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:43.389198+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:43.389198+0000","last_peered":"2024-09-05T13:18:43.389198+0000","last_clean":"2024-09-05T13:18:43.389198+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:43.389198+0000","last_undegraded":"2024-09-05T13:18:43.389198+0000","last_fullsized":"2024-09-05T13:18:43.389198+0000","mapping_epoch":44,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11566,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13810","reported_seq":13934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:43.403714+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:43.403714+0000","last_peered":"2024-09-05T13:18:43.403714+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:43.403714+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11319,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":871},{"shards":"2","objects":2385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13998","reported_seq":14135,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:44.270405+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:44.270405+0000","last_peered":"2024-09-05T13:18:44.270405+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:44.270405+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":6880,"num_object_clones":0,"num_object_copies":13760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10149,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2461},{"shards":"0,2","objects":861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14147","reported_seq":14335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:44.244416+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:44.244416+0000","last_peered":"2024-09-05T13:18:44.244416+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:44.244416+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28495872,"num_objects":6957,"num_object_clones":0,"num_object_copies":13914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10046,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2513},{"shards":"0,2","objects":853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14222","reported_seq":14362,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:44.277310+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:44.277310+0000","last_peered":"2024-09-05T13:18:44.277310+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:44.277310+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090368,"num_objects":6858,"num_object_clones":0,"num_object_copies":13716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10352,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2292},{"shards":"0,2","objects":886}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14248","reported_seq":14389,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:44.287522+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:44.287522+0000","last_peered":"2024-09-05T13:18:44.287522+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:44.287522+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28443107,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10916,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2380},{"shards":"0,2","objects":919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452993531,"num_objects":110596,"num_object_clones":0,"num_object_copies":221192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45306,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":110596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180591,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7315,"num_bytes_recovered":29945856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1085329408,"data_stored":1085245391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664024,"kb_used_data":365768,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93707816,"statfs":{"total":96636764160,"available":95956803584,"internally_reserved":0,"allocated":374546432,"data_stored":374286875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803976,"kb_used_data":403064,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93567864,"statfs":{"total":96636764160,"available":95813492736,"internally_reserved":0,"allocated":412737536,"data_stored":412477974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,9,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618120,"kb_used_data":293496,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93753720,"statfs":{"total":96636764160,"available":96003809280,"internally_reserved":0,"allocated":300539904,"data_stored":300211236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411975680,"data_stored":411972067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300077056,"data_stored":300004337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":373276672,"data_stored":373268987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:49.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:50.051 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:50.052 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:50.076 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-05T13:18:49.657554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":444019214,"num_objects":108407,"num_object_clones":0,"num_object_copies":216814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45327,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":108407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182784,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7443,"num_bytes_recovered":30466067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2058160,"kb_used_data":1051008,"kb_used_omap":12,"kb_used_meta":1007091,"kb_avail":281057360,"statfs":{"total":289910292480,"available":287802736640,"internally_reserved":0,"allocated":1076232192,"data_stored":1075384405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1031261190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,4,4,6,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27316224,"num_objects":-6669,"num_object_clones":0,"num_object_copies":-13338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":-415,"num_objects_unfound":0,"num_objects_dirty":-6669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":371,"num_bytes_recovered":1519616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005759"},"pg_stats":[{"pgid":"2.f","version":"45'14133","reported_seq":14208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.438980+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:49.438980+0000","last_peered":"2024-09-05T13:18:49.438980+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:49.438980+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14133,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2418},{"shards":"0,2","objects":773}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14069","reported_seq":14172,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.412699+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:48.412699+0000","last_peered":"2024-09-05T13:18:48.412699+0000","last_clean":"2024-09-05T13:18:48.412699+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:48.412699+0000","last_undegraded":"2024-09-05T13:18:48.412699+0000","last_fullsized":"2024-09-05T13:18:48.412699+0000","mapping_epoch":35,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27381760,"num_objects":6685,"num_object_clones":0,"num_object_copies":13370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12079,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14354","reported_seq":14483,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.225855+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:48.225855+0000","last_peered":"2024-09-05T13:18:48.225855+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:48.225855+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28020760,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12165,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":738},{"shards":"2","objects":2350}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14155","reported_seq":14225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.296148+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:48.296148+0000","last_peered":"2024-09-05T13:18:48.296148+0000","last_clean":"2024-09-05T13:18:48.296148+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:48.296148+0000","last_undegraded":"2024-09-05T13:18:48.296148+0000","last_fullsized":"2024-09-05T13:18:48.296148+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28143616,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14155,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14142","reported_seq":14277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:49.297992+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:49.297992+0000","last_peered":"2024-09-05T13:18:49.297992+0000","last_clean":"2024-09-05T13:18:49.297992+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:49.297992+0000","last_undegraded":"2024-09-05T13:18:49.297992+0000","last_fullsized":"2024-09-05T13:18:49.297992+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10682,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14299","reported_seq":14438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.316513+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:49.316513+0000","last_peered":"2024-09-05T13:18:49.316513+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:49.316513+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28082176,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10754,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2405},{"shards":"0,2","objects":732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14017","reported_seq":20259,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.352669+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:49.352669+0000","last_peered":"2024-09-05T13:18:49.352669+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:49.352669+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27078656,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3050,"num_bytes_recovered":12484608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14289","reported_seq":14422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:49.274789+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:49.274789+0000","last_peered":"2024-09-05T13:18:49.274789+0000","last_clean":"2024-09-05T13:18:49.274789+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:49.274789+0000","last_undegraded":"2024-09-05T13:18:49.274789+0000","last_fullsized":"2024-09-05T13:18:49.274789+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27914240,"num_objects":6815,"num_object_clones":0,"num_object_copies":13630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10899,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14248","reported_seq":14384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.361398+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:48.361398+0000","last_peered":"2024-09-05T13:18:48.361398+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:48.361398+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28037120,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11902,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":753},{"shards":"2","objects":2394}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14157","reported_seq":14295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.349947+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:49.349947+0000","last_peered":"2024-09-05T13:18:49.349947+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:49.349947+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27660288,"num_objects":6753,"num_object_clones":0,"num_object_copies":13506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10524,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2299},{"shards":"0,2","objects":756}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14067","reported_seq":22969,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.336859+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:48.336859+0000","last_peered":"2024-09-05T13:18:48.336859+0000","last_clean":"2024-09-05T13:18:48.336859+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:48.336859+0000","last_undegraded":"2024-09-05T13:18:48.336859+0000","last_fullsized":"2024-09-05T13:18:48.336859+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11662,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13932","reported_seq":14056,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.369100+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:48.369100+0000","last_peered":"2024-09-05T13:18:48.369100+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:48.369100+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11441,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":2263}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14151","reported_seq":14292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.378480+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:49.378480+0000","last_peered":"2024-09-05T13:18:49.378480+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:49.378480+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27553792,"num_objects":6727,"num_object_clones":0,"num_object_copies":13454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14301","reported_seq":14489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.349196+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:49.349196+0000","last_peered":"2024-09-05T13:18:49.349196+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:49.349196+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27865088,"num_objects":6803,"num_object_clones":0,"num_object_copies":13606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2359},{"shards":"0,2","objects":699}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14359","reported_seq":14499,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.319361+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:49.319361+0000","last_peered":"2024-09-05T13:18:49.319361+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:49.319361+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529216,"num_objects":6721,"num_object_clones":0,"num_object_copies":13442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10489,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2155},{"shards":"0,2","objects":749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14391","reported_seq":14532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.335270+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:49.335270+0000","last_peered":"2024-09-05T13:18:49.335270+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:49.335270+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27857379,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11059,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":444019195,"num_objects":108405,"num_object_clones":0,"num_object_copies":216810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45327,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":108405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182782,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7442,"num_bytes_recovered":30466048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073696768,"data_stored":1073612751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664024,"kb_used_data":361608,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93707816,"statfs":{"total":96636764160,"available":95956803584,"internally_reserved":0,"allocated":370286592,"data_stored":370027035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784732,"kb_used_data":396300,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93587108,"statfs":{"total":96636764160,"available":95833198592,"internally_reserved":0,"allocated":405811200,"data_stored":405551638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,3,0,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609404,"kb_used_data":293100,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93762436,"statfs":{"total":96636764160,"available":96012734464,"internally_reserved":0,"allocated":300134400,"data_stored":299805732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":404942848,"data_stored":404939235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299704320,"data_stored":299631601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369049600,"data_stored":369041915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:50.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:50.647 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:50.649 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:50.678 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-05T13:18:49.657554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":444019214,"num_objects":108407,"num_object_clones":0,"num_object_copies":216814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45327,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":108407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182784,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7443,"num_bytes_recovered":30466067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2058160,"kb_used_data":1051008,"kb_used_omap":12,"kb_used_meta":1007091,"kb_avail":281057360,"statfs":{"total":289910292480,"available":287802736640,"internally_reserved":0,"allocated":1076232192,"data_stored":1075384405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1031261190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,4,4,6,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27316224,"num_objects":-6669,"num_object_clones":0,"num_object_copies":-13338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":-415,"num_objects_unfound":0,"num_objects_dirty":-6669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":371,"num_bytes_recovered":1519616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005759"},"pg_stats":[{"pgid":"2.f","version":"45'14133","reported_seq":14208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.438980+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:49.438980+0000","last_peered":"2024-09-05T13:18:49.438980+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:49.438980+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14133,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2418},{"shards":"0,2","objects":773}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14069","reported_seq":14172,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.412699+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:48.412699+0000","last_peered":"2024-09-05T13:18:48.412699+0000","last_clean":"2024-09-05T13:18:48.412699+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:48.412699+0000","last_undegraded":"2024-09-05T13:18:48.412699+0000","last_fullsized":"2024-09-05T13:18:48.412699+0000","mapping_epoch":35,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27381760,"num_objects":6685,"num_object_clones":0,"num_object_copies":13370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12079,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14354","reported_seq":14483,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.225855+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:48.225855+0000","last_peered":"2024-09-05T13:18:48.225855+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:48.225855+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28020760,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12165,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":738},{"shards":"2","objects":2350}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14155","reported_seq":14225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.296148+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:48.296148+0000","last_peered":"2024-09-05T13:18:48.296148+0000","last_clean":"2024-09-05T13:18:48.296148+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:48.296148+0000","last_undegraded":"2024-09-05T13:18:48.296148+0000","last_fullsized":"2024-09-05T13:18:48.296148+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28143616,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14155,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14142","reported_seq":14277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:49.297992+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:49.297992+0000","last_peered":"2024-09-05T13:18:49.297992+0000","last_clean":"2024-09-05T13:18:49.297992+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:49.297992+0000","last_undegraded":"2024-09-05T13:18:49.297992+0000","last_fullsized":"2024-09-05T13:18:49.297992+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10682,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14299","reported_seq":14438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.316513+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:49.316513+0000","last_peered":"2024-09-05T13:18:49.316513+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:49.316513+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28082176,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10754,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2405},{"shards":"0,2","objects":732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14017","reported_seq":20259,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.352669+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:49.352669+0000","last_peered":"2024-09-05T13:18:49.352669+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:49.352669+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27078656,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3050,"num_bytes_recovered":12484608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14289","reported_seq":14422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:49.274789+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:49.274789+0000","last_peered":"2024-09-05T13:18:49.274789+0000","last_clean":"2024-09-05T13:18:49.274789+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:49.274789+0000","last_undegraded":"2024-09-05T13:18:49.274789+0000","last_fullsized":"2024-09-05T13:18:49.274789+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27914240,"num_objects":6815,"num_object_clones":0,"num_object_copies":13630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10899,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14248","reported_seq":14384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.361398+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:48.361398+0000","last_peered":"2024-09-05T13:18:48.361398+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:48.361398+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28037120,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11902,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":753},{"shards":"2","objects":2394}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14157","reported_seq":14295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.349947+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:49.349947+0000","last_peered":"2024-09-05T13:18:49.349947+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:49.349947+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27660288,"num_objects":6753,"num_object_clones":0,"num_object_copies":13506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10524,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2299},{"shards":"0,2","objects":756}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14067","reported_seq":22969,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.336859+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:48.336859+0000","last_peered":"2024-09-05T13:18:48.336859+0000","last_clean":"2024-09-05T13:18:48.336859+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:48.336859+0000","last_undegraded":"2024-09-05T13:18:48.336859+0000","last_fullsized":"2024-09-05T13:18:48.336859+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11662,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13932","reported_seq":14056,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.369100+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:48.369100+0000","last_peered":"2024-09-05T13:18:48.369100+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:48.369100+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11441,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":2263}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14151","reported_seq":14292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.378480+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:49.378480+0000","last_peered":"2024-09-05T13:18:49.378480+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:49.378480+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27553792,"num_objects":6727,"num_object_clones":0,"num_object_copies":13454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14301","reported_seq":14489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.349196+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:49.349196+0000","last_peered":"2024-09-05T13:18:49.349196+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:49.349196+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27865088,"num_objects":6803,"num_object_clones":0,"num_object_copies":13606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2359},{"shards":"0,2","objects":699}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14359","reported_seq":14499,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.319361+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:49.319361+0000","last_peered":"2024-09-05T13:18:49.319361+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:49.319361+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529216,"num_objects":6721,"num_object_clones":0,"num_object_copies":13442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10489,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2155},{"shards":"0,2","objects":749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14391","reported_seq":14532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.335270+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:49.335270+0000","last_peered":"2024-09-05T13:18:49.335270+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:49.335270+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27857379,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11059,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":444019195,"num_objects":108405,"num_object_clones":0,"num_object_copies":216810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45327,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":108405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182782,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7442,"num_bytes_recovered":30466048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073696768,"data_stored":1073612751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664024,"kb_used_data":361608,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93707816,"statfs":{"total":96636764160,"available":95956803584,"internally_reserved":0,"allocated":370286592,"data_stored":370027035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784732,"kb_used_data":396300,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93587108,"statfs":{"total":96636764160,"available":95833198592,"internally_reserved":0,"allocated":405811200,"data_stored":405551638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,3,0,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609404,"kb_used_data":293100,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93762436,"statfs":{"total":96636764160,"available":96012734464,"internally_reserved":0,"allocated":300134400,"data_stored":299805732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":404942848,"data_stored":404939235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299704320,"data_stored":299631601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369049600,"data_stored":369041915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:50.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:50.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'14133", 'reported_seq': 14208, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:49.438980+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:49.438980+0000', 'last_peered': '2024-09-05T13:18:49.438980+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:49.438980+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28012544, 'num_objects': 6839, 'num_object_clones': 0, 'num_object_copies': 13678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14133, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2418}, {'shards': '0,2', 'objects': 773}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:50.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:50.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'14354", 'reported_seq': 14483, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:48.225855+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:48.225855+0000', 'last_peered': '2024-09-05T13:18:48.225855+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:48.225855+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28020760, 'num_objects': 6842, 'num_object_clones': 0, 'num_object_copies': 13684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6842, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12165, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 738}, {'shards': '2', 'objects': 2350}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:50.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:50.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'14299", 'reported_seq': 14438, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:49.316513+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:49.316513+0000', 'last_peered': '2024-09-05T13:18:49.316513+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:49.316513+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28082176, 'num_objects': 6856, 'num_object_clones': 0, 'num_object_copies': 13712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6856, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10754, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2405}, {'shards': '0,2', 'objects': 732}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:50.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:50.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'14017", 'reported_seq': 20259, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:49.352669+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:49.352669+0000', 'last_peered': '2024-09-05T13:18:49.352669+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:49.352669+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27078656, 'num_objects': 6611, 'num_object_clones': 0, 'num_object_copies': 13222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 659, 'num_objects_misplaced': 729, 'num_objects_unfound': 0, 'num_objects_dirty': 6611, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10336, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3050, 'num_bytes_recovered': 12484608, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5223}, {'shards': '0,2', 'objects': 729}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:50.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:50.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'14248", 'reported_seq': 14384, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:48.361398+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:48.361398+0000', 'last_peered': '2024-09-05T13:18:48.361398+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:48.361398+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28037120, 'num_objects': 6845, 'num_object_clones': 0, 'num_object_copies': 13690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11902, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 753}, {'shards': '2', 'objects': 2394}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:50.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:50.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'14157", 'reported_seq': 14295, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:49.349947+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:49.349947+0000', 'last_peered': '2024-09-05T13:18:49.349947+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:49.349947+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27660288, 'num_objects': 6753, 'num_object_clones': 0, 'num_object_copies': 13506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10524, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2299}, {'shards': '0,2', 'objects': 756}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:50.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:50.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'13932", 'reported_seq': 14056, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:48.369100+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:48.369100+0000', 'last_peered': '2024-09-05T13:18:48.369100+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:48.369100+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "36'3900", 'ondisk_log_start': "36'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27238400, 'num_objects': 6650, 'num_object_clones': 0, 'num_object_copies': 13300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11441, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 749}, {'shards': '2', 'objects': 2263}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:50.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:50.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'14151", 'reported_seq': 14292, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:49.378480+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:49.378480+0000', 'last_peered': '2024-09-05T13:18:49.378480+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:49.378480+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27553792, 'num_objects': 6727, 'num_object_clones': 0, 'num_object_copies': 13454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10302, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2308}, {'shards': '0,2', 'objects': 708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:50.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:50.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'14301", 'reported_seq': 14489, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:49.349196+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:49.349196+0000', 'last_peered': '2024-09-05T13:18:49.349196+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:49.349196+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27865088, 'num_objects': 6803, 'num_object_clones': 0, 'num_object_copies': 13606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10200, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2359}, {'shards': '0,2', 'objects': 699}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:50.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:50.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'14359", 'reported_seq': 14499, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:49.319361+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:49.319361+0000', 'last_peered': '2024-09-05T13:18:49.319361+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:49.319361+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27529216, 'num_objects': 6721, 'num_object_clones': 0, 'num_object_copies': 13442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10489, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2155}, {'shards': '0,2', 'objects': 749}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:50.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:50.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'14391", 'reported_seq': 14532, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:49.335270+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:49.335270+0000', 'last_peered': '2024-09-05T13:18:49.335270+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:49.335270+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27857379, 'num_objects': 6802, 'num_object_clones': 0, 'num_object_copies': 13604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11059, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2237}, {'shards': '0,2', 'objects': 776}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:50.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:51.357 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:51.357 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45327/216814 objects degraded (20.906%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":458,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":108407,"data_bytes":444019214,"bytes_used":2107555840,"bytes_avail":287802736640,"bytes_total":289910292480,"degraded_objects":45327,"degraded_total":216814,"degraded_ratio":0.20905937808444106,"misplaced_objects":729,"misplaced_total":216814,"misplaced_ratio":0.0033623290008947763,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":126573,"recovering_keys_per_sec":0,"num_objects_recovered":371,"num_bytes_recovered":1519616,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":555},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:51.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:51.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:51.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:51.950 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-05T13:18:51.658084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":444019214,"num_objects":108407,"num_object_clones":0,"num_object_copies":216814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45327,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":108407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182784,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7443,"num_bytes_recovered":30466067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2058160,"kb_used_data":1051008,"kb_used_omap":12,"kb_used_meta":1007091,"kb_avail":281057360,"statfs":{"total":289910292480,"available":287802736640,"internally_reserved":0,"allocated":1076232192,"data_stored":1075384405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1031261190},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,4,4,6,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18092032,"num_objects":-4417,"num_object_clones":0,"num_object_copies":-8834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":-293,"num_objects_unfound":0,"num_objects_dirty":-4417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4417,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003876"},"pg_stats":[{"pgid":"2.f","version":"45'14133","reported_seq":14208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.438980+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:49.438980+0000","last_peered":"2024-09-05T13:18:49.438980+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:49.438980+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14133,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2418},{"shards":"0,2","objects":773}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14069","reported_seq":14172,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.412699+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:48.412699+0000","last_peered":"2024-09-05T13:18:48.412699+0000","last_clean":"2024-09-05T13:18:48.412699+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:48.412699+0000","last_undegraded":"2024-09-05T13:18:48.412699+0000","last_fullsized":"2024-09-05T13:18:48.412699+0000","mapping_epoch":35,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27381760,"num_objects":6685,"num_object_clones":0,"num_object_copies":13370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12079,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14354","reported_seq":14483,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.225855+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:48.225855+0000","last_peered":"2024-09-05T13:18:48.225855+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:48.225855+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28020760,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12165,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":738},{"shards":"2","objects":2350}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14155","reported_seq":14225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.296148+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:48.296148+0000","last_peered":"2024-09-05T13:18:48.296148+0000","last_clean":"2024-09-05T13:18:48.296148+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:48.296148+0000","last_undegraded":"2024-09-05T13:18:48.296148+0000","last_fullsized":"2024-09-05T13:18:48.296148+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28143616,"num_objects":6871,"num_object_clones":0,"num_object_copies":13742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14155,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14142","reported_seq":14277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:49.297992+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:49.297992+0000","last_peered":"2024-09-05T13:18:49.297992+0000","last_clean":"2024-09-05T13:18:49.297992+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:49.297992+0000","last_undegraded":"2024-09-05T13:18:49.297992+0000","last_fullsized":"2024-09-05T13:18:49.297992+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10682,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14299","reported_seq":14438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.316513+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:49.316513+0000","last_peered":"2024-09-05T13:18:49.316513+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:49.316513+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28082176,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10754,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2405},{"shards":"0,2","objects":732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14017","reported_seq":20259,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.352669+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:49.352669+0000","last_peered":"2024-09-05T13:18:49.352669+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:49.352669+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27078656,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3050,"num_bytes_recovered":12484608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14289","reported_seq":14422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:49.274789+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:49.274789+0000","last_peered":"2024-09-05T13:18:49.274789+0000","last_clean":"2024-09-05T13:18:49.274789+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:49.274789+0000","last_undegraded":"2024-09-05T13:18:49.274789+0000","last_fullsized":"2024-09-05T13:18:49.274789+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27914240,"num_objects":6815,"num_object_clones":0,"num_object_copies":13630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10899,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14248","reported_seq":14384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.361398+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:48.361398+0000","last_peered":"2024-09-05T13:18:48.361398+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:48.361398+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28037120,"num_objects":6845,"num_object_clones":0,"num_object_copies":13690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11902,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":753},{"shards":"2","objects":2394}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14157","reported_seq":14295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.349947+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:49.349947+0000","last_peered":"2024-09-05T13:18:49.349947+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:49.349947+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27660288,"num_objects":6753,"num_object_clones":0,"num_object_copies":13506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10524,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2299},{"shards":"0,2","objects":756}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14067","reported_seq":22969,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:48.336859+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:48.336859+0000","last_peered":"2024-09-05T13:18:48.336859+0000","last_clean":"2024-09-05T13:18:48.336859+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:48.336859+0000","last_undegraded":"2024-09-05T13:18:48.336859+0000","last_fullsized":"2024-09-05T13:18:48.336859+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11662,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13932","reported_seq":14056,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:48.369100+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:48.369100+0000","last_peered":"2024-09-05T13:18:48.369100+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:48.369100+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"36'3900","ondisk_log_start":"36'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238400,"num_objects":6650,"num_object_clones":0,"num_object_copies":13300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11441,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":2263}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14151","reported_seq":14292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.378480+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:49.378480+0000","last_peered":"2024-09-05T13:18:49.378480+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:49.378480+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27553792,"num_objects":6727,"num_object_clones":0,"num_object_copies":13454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14301","reported_seq":14489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.349196+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:49.349196+0000","last_peered":"2024-09-05T13:18:49.349196+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:49.349196+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27865088,"num_objects":6803,"num_object_clones":0,"num_object_copies":13606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2359},{"shards":"0,2","objects":699}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14359","reported_seq":14499,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.319361+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:49.319361+0000","last_peered":"2024-09-05T13:18:49.319361+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:49.319361+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529216,"num_objects":6721,"num_object_clones":0,"num_object_copies":13442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10489,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2155},{"shards":"0,2","objects":749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14391","reported_seq":14532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.335270+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:49.335270+0000","last_peered":"2024-09-05T13:18:49.335270+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:49.335270+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27857379,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11059,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":444019195,"num_objects":108405,"num_object_clones":0,"num_object_copies":216810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45327,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":108405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182782,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7442,"num_bytes_recovered":30466048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073696768,"data_stored":1073612751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664024,"kb_used_data":361608,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93707816,"statfs":{"total":96636764160,"available":95956803584,"internally_reserved":0,"allocated":370286592,"data_stored":370027035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784732,"kb_used_data":396300,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93587108,"statfs":{"total":96636764160,"available":95833198592,"internally_reserved":0,"allocated":405811200,"data_stored":405551638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,3,0,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609404,"kb_used_data":293100,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93762436,"statfs":{"total":96636764160,"available":96012734464,"internally_reserved":0,"allocated":300134400,"data_stored":299805732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":404942848,"data_stored":404939235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299704320,"data_stored":299631601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369049600,"data_stored":369041915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:54.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:55.559 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:55.581 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-05T13:18:53.658851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":440541710,"num_objects":107558,"num_object_clones":0,"num_object_copies":215116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45327,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":107558,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183633,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7443,"num_bytes_recovered":30466067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2061420,"kb_used_data":1045932,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281054100,"statfs":{"total":289910292480,"available":287799398400,"internally_reserved":0,"allocated":1071034368,"data_stored":1070186581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1039780870},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,7,8,10,6,13,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21569536,"num_objects":-5266,"num_object_clones":0,"num_object_copies":-10532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":-293,"num_objects_unfound":0,"num_objects_dirty":-5266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5266,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003840"},"pg_stats":[{"pgid":"2.f","version":"45'14133","reported_seq":14208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.438980+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:49.438980+0000","last_peered":"2024-09-05T13:18:49.438980+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:49.438980+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14133,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2418},{"shards":"0,2","objects":773}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14200","reported_seq":14305,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.413660+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:53.413660+0000","last_peered":"2024-09-05T13:18:53.413660+0000","last_clean":"2024-09-05T13:18:53.413660+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:53.413660+0000","last_undegraded":"2024-09-05T13:18:53.413660+0000","last_fullsized":"2024-09-05T13:18:53.413660+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12210,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14503","reported_seq":14630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.356957+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:53.356957+0000","last_peered":"2024-09-05T13:18:53.356957+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:53.356957+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410456,"num_objects":6693,"num_object_clones":0,"num_object_copies":13386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12314,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":589},{"shards":"2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14304","reported_seq":14374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.413669+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:53.413669+0000","last_peered":"2024-09-05T13:18:53.413669+0000","last_clean":"2024-09-05T13:18:53.413669+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:53.413669+0000","last_undegraded":"2024-09-05T13:18:53.413669+0000","last_fullsized":"2024-09-05T13:18:53.413669+0000","mapping_epoch":35,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27533312,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14304,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14142","reported_seq":14277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:49.297992+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:49.297992+0000","last_peered":"2024-09-05T13:18:49.297992+0000","last_clean":"2024-09-05T13:18:49.297992+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:49.297992+0000","last_undegraded":"2024-09-05T13:18:49.297992+0000","last_fullsized":"2024-09-05T13:18:49.297992+0000","mapping_epoch":34,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10682,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14299","reported_seq":14438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.316513+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:49.316513+0000","last_peered":"2024-09-05T13:18:49.316513+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:49.316513+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28082176,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10754,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2405},{"shards":"0,2","objects":732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14017","reported_seq":20259,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.352669+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:49.352669+0000","last_peered":"2024-09-05T13:18:49.352669+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:49.352669+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27078656,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10336,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3050,"num_bytes_recovered":12484608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14289","reported_seq":14422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:49.274789+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:49.274789+0000","last_peered":"2024-09-05T13:18:49.274789+0000","last_clean":"2024-09-05T13:18:49.274789+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:49.274789+0000","last_undegraded":"2024-09-05T13:18:49.274789+0000","last_fullsized":"2024-09-05T13:18:49.274789+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27914240,"num_objects":6815,"num_object_clones":0,"num_object_copies":13630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10899,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14383","reported_seq":14519,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.354314+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:53.354314+0000","last_peered":"2024-09-05T13:18:53.354314+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:53.354314+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27484160,"num_objects":6710,"num_object_clones":0,"num_object_copies":13420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12037,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":618},{"shards":"2","objects":2259}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14157","reported_seq":14295,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.349947+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:49.349947+0000","last_peered":"2024-09-05T13:18:49.349947+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:49.349947+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27660288,"num_objects":6753,"num_object_clones":0,"num_object_copies":13506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10524,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2299},{"shards":"0,2","objects":756}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14217","reported_seq":23119,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.433590+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:53.433590+0000","last_peered":"2024-09-05T13:18:53.433590+0000","last_clean":"2024-09-05T13:18:53.433590+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:53.433590+0000","last_undegraded":"2024-09-05T13:18:53.433590+0000","last_fullsized":"2024-09-05T13:18:53.433590+0000","mapping_epoch":44,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11812,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14067","reported_seq":14191,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.317958+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:53.317958+0000","last_peered":"2024-09-05T13:18:53.317958+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:53.317958+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26685440,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11576,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":614},{"shards":"2","objects":2128}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14151","reported_seq":14292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.378480+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:49.378480+0000","last_peered":"2024-09-05T13:18:49.378480+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:49.378480+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27553792,"num_objects":6727,"num_object_clones":0,"num_object_copies":13454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14301","reported_seq":14489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.349196+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:49.349196+0000","last_peered":"2024-09-05T13:18:49.349196+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:49.349196+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27865088,"num_objects":6803,"num_object_clones":0,"num_object_copies":13606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10200,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2359},{"shards":"0,2","objects":699}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14359","reported_seq":14499,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.319361+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:49.319361+0000","last_peered":"2024-09-05T13:18:49.319361+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:49.319361+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27529216,"num_objects":6721,"num_object_clones":0,"num_object_copies":13442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10489,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2155},{"shards":"0,2","objects":749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14391","reported_seq":14532,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:49.335270+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:49.335270+0000","last_peered":"2024-09-05T13:18:49.335270+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:49.335270+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27857379,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11059,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":776}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":440541691,"num_objects":107556,"num_object_clones":0,"num_object_copies":215112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45327,"num_objects_misplaced":729,"num_objects_unfound":0,"num_objects_dirty":107556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183631,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7442,"num_bytes_recovered":30466048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068429312,"data_stored":1068345295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663696,"kb_used_data":357104,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93708144,"statfs":{"total":96636764160,"available":95957139456,"internally_reserved":0,"allocated":365674496,"data_stored":365414939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784732,"kb_used_data":396300,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93587108,"statfs":{"total":96636764160,"available":95833198592,"internally_reserved":0,"allocated":405811200,"data_stored":405551638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,3,0,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612992,"kb_used_data":292528,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93758848,"statfs":{"total":96636764160,"available":96009060352,"internally_reserved":0,"allocated":299548672,"data_stored":299220004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,4,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":404942848,"data_stored":404939235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299073536,"data_stored":299000817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364412928,"data_stored":364405243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:55.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:56.187 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:56.188 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:56.208 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-05T13:18:55.659711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":434934286,"num_objects":106189,"num_object_clones":0,"num_object_copies":212378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":576,"num_objects_unfound":0,"num_objects_dirty":106189,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185002,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7579,"num_bytes_recovered":31023123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160885,"ondisk_log_size":160885,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2063328,"kb_used_data":1039516,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281052192,"statfs":{"total":289910292480,"available":287797444608,"internally_reserved":0,"allocated":1064464384,"data_stored":1063616597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1048300550},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,5,8,15,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23941120,"num_objects":-5845,"num_object_clones":0,"num_object_copies":-11690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":61,"num_objects_misplaced":-446,"num_objects_unfound":0,"num_objects_dirty":-5845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5845,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":1576960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004091"},"pg_stats":[{"pgid":"2.f","version":"45'14280","reported_seq":14356,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.398679+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:54.398679+0000","last_peered":"2024-09-05T13:18:54.398679+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:54.398679+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410432,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14280,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2271},{"shards":"0,2","objects":626}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14200","reported_seq":14305,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.413660+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:53.413660+0000","last_peered":"2024-09-05T13:18:53.413660+0000","last_clean":"2024-09-05T13:18:53.413660+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:53.413660+0000","last_undegraded":"2024-09-05T13:18:53.413660+0000","last_fullsized":"2024-09-05T13:18:53.413660+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12210,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14503","reported_seq":14630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.356957+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:53.356957+0000","last_peered":"2024-09-05T13:18:53.356957+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:53.356957+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410456,"num_objects":6693,"num_object_clones":0,"num_object_copies":13386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12314,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":589},{"shards":"2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14304","reported_seq":14374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.413669+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:53.413669+0000","last_peered":"2024-09-05T13:18:53.413669+0000","last_clean":"2024-09-05T13:18:53.413669+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:53.413669+0000","last_undegraded":"2024-09-05T13:18:53.413669+0000","last_fullsized":"2024-09-05T13:18:53.413669+0000","mapping_epoch":35,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27533312,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14304,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14275","reported_seq":14410,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:54.424605+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:54.424605+0000","last_peered":"2024-09-05T13:18:54.424605+0000","last_clean":"2024-09-05T13:18:54.424605+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:54.424605+0000","last_undegraded":"2024-09-05T13:18:54.424605+0000","last_fullsized":"2024-09-05T13:18:54.424605+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27066368,"num_objects":6608,"num_object_clones":0,"num_object_copies":13216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10815,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14438","reported_seq":14576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.489303+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:54.489303+0000","last_peered":"2024-09-05T13:18:54.489303+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:54.489303+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27512832,"num_objects":6717,"num_object_clones":0,"num_object_copies":13434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10893,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2266},{"shards":"0,2","objects":593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14158","reported_seq":20672,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.472802+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:54.472802+0000","last_peered":"2024-09-05T13:18:54.472802+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:54.472802+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":576,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10477,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3186,"num_bytes_recovered":13041664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14430","reported_seq":14563,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:54.320738+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:54.320738+0000","last_peered":"2024-09-05T13:18:54.320738+0000","last_clean":"2024-09-05T13:18:54.320738+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:54.320738+0000","last_undegraded":"2024-09-05T13:18:54.320738+0000","last_fullsized":"2024-09-05T13:18:54.320738+0000","mapping_epoch":34,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27336704,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11040,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14383","reported_seq":14519,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.354314+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:53.354314+0000","last_peered":"2024-09-05T13:18:53.354314+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:53.354314+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27484160,"num_objects":6710,"num_object_clones":0,"num_object_copies":13420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12037,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":618},{"shards":"2","objects":2259}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14296","reported_seq":14434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.435006+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:54.435006+0000","last_peered":"2024-09-05T13:18:54.435006+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:54.435006+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10663,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2160},{"shards":"0,2","objects":617}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14217","reported_seq":23119,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.433590+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:53.433590+0000","last_peered":"2024-09-05T13:18:53.433590+0000","last_clean":"2024-09-05T13:18:53.433590+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:53.433590+0000","last_undegraded":"2024-09-05T13:18:53.433590+0000","last_fullsized":"2024-09-05T13:18:53.433590+0000","mapping_epoch":44,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11812,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14067","reported_seq":14191,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.317958+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:53.317958+0000","last_peered":"2024-09-05T13:18:53.317958+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:53.317958+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26685440,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11576,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":614},{"shards":"2","objects":2128}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14287","reported_seq":14426,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.418118+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:54.418118+0000","last_peered":"2024-09-05T13:18:54.418118+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:54.418118+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26996736,"num_objects":6591,"num_object_clones":0,"num_object_copies":13182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10438,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2172},{"shards":"0,2","objects":572}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14414","reported_seq":14602,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.458108+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:54.458108+0000","last_peered":"2024-09-05T13:18:54.458108+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:54.458108+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10313,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2246},{"shards":"0,2","objects":586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14488","reported_seq":14626,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.344586+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:54.344586+0000","last_peered":"2024-09-05T13:18:54.344586+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:54.344586+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27000832,"num_objects":6592,"num_object_clones":0,"num_object_copies":13184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10618,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026},{"shards":"0,2","objects":620}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14542","reported_seq":14683,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.357907+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:54.357907+0000","last_peered":"2024-09-05T13:18:54.357907+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:54.357907+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"40'4500","ondisk_log_start":"40'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238883,"num_objects":6651,"num_object_clones":0,"num_object_copies":13302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11210,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2086},{"shards":"0,2","objects":625}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":434934267,"num_objects":106187,"num_object_clones":0,"num_object_copies":212374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":576,"num_objects_unfound":0,"num_objects_dirty":106187,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185000,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7578,"num_bytes_recovered":31023104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1061740544,"data_stored":1061656527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160882,"ondisk_log_size":160882,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663696,"kb_used_data":357104,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93708144,"statfs":{"total":96636764160,"available":95957139456,"internally_reserved":0,"allocated":365674496,"data_stored":365414939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":786640,"kb_used_data":389884,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93585200,"statfs":{"total":96636764160,"available":95831244800,"internally_reserved":0,"allocated":399241216,"data_stored":398981654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612992,"kb_used_data":292528,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93758848,"statfs":{"total":96636764160,"available":96009060352,"internally_reserved":0,"allocated":299548672,"data_stored":299220004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,4,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398254080,"data_stored":398250467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299073536,"data_stored":299000817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364412928,"data_stored":364405243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:18:56.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:18:56.209 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'14280", 'reported_seq': 14356, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:54.398679+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:54.398679+0000', 'last_peered': '2024-09-05T13:18:54.398679+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:54.398679+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27410432, 'num_objects': 6692, 'num_object_clones': 0, 'num_object_copies': 13384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14280, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2271}, {'shards': '0,2', 'objects': 626}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'14503", 'reported_seq': 14630, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:53.356957+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:53.356957+0000', 'last_peered': '2024-09-05T13:18:53.356957+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:53.356957+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27410456, 'num_objects': 6693, 'num_object_clones': 0, 'num_object_copies': 13386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6693, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12314, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 589}, {'shards': '2', 'objects': 2201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'14438", 'reported_seq': 14576, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:54.489303+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:54.489303+0000', 'last_peered': '2024-09-05T13:18:54.489303+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:54.489303+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "40'4400", 'ondisk_log_start': "40'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27512832, 'num_objects': 6717, 'num_object_clones': 0, 'num_object_copies': 13434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10893, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2266}, {'shards': '0,2', 'objects': 593}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'14158", 'reported_seq': 20672, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:54.472802+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:54.472802+0000', 'last_peered': '2024-09-05T13:18:54.472802+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:54.472802+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26501120, 'num_objects': 6470, 'num_object_clones': 0, 'num_object_copies': 12940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 676, 'num_objects_misplaced': 576, 'num_objects_unfound': 0, 'num_objects_dirty': 6470, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10477, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3186, 'num_bytes_recovered': 13041664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5218}, {'shards': '0,2', 'objects': 576}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'14383", 'reported_seq': 14519, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:53.354314+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:53.354314+0000', 'last_peered': '2024-09-05T13:18:53.354314+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:53.354314+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27484160, 'num_objects': 6710, 'num_object_clones': 0, 'num_object_copies': 13420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12037, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 618}, {'shards': '2', 'objects': 2259}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:18:56.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'14296", 'reported_seq': 14434, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:54.435006+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:54.435006+0000', 'last_peered': '2024-09-05T13:18:54.435006+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:54.435006+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27090944, 'num_objects': 6614, 'num_object_clones': 0, 'num_object_copies': 13228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10663, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2160}, {'shards': '0,2', 'objects': 617}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:56.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:18:56.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'14067", 'reported_seq': 14191, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:53.317958+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:53.317958+0000', 'last_peered': '2024-09-05T13:18:53.317958+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:53.317958+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26685440, 'num_objects': 6515, 'num_object_clones': 0, 'num_object_copies': 13030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11576, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 614}, {'shards': '2', 'objects': 2128}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:18:56.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:18:56.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'14287", 'reported_seq': 14426, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:54.418118+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:54.418118+0000', 'last_peered': '2024-09-05T13:18:54.418118+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:54.418118+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26996736, 'num_objects': 6591, 'num_object_clones': 0, 'num_object_copies': 13182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10438, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2172}, {'shards': '0,2', 'objects': 572}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:56.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:18:56.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'14414", 'reported_seq': 14602, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:54.458108+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:54.458108+0000', 'last_peered': '2024-09-05T13:18:54.458108+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:54.458108+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "40'4400", 'ondisk_log_start': "40'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27402240, 'num_objects': 6690, 'num_object_clones': 0, 'num_object_copies': 13380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10313, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2246}, {'shards': '0,2', 'objects': 586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:56.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:18:56.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'14488", 'reported_seq': 14626, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:54.344586+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:54.344586+0000', 'last_peered': '2024-09-05T13:18:54.344586+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:54.344586+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "39'4400", 'ondisk_log_start': "39'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27000832, 'num_objects': 6592, 'num_object_clones': 0, 'num_object_copies': 13184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10618, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2026}, {'shards': '0,2', 'objects': 620}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:56.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:18:56.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'14542", 'reported_seq': 14683, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:54.357907+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:54.357907+0000', 'last_peered': '2024-09-05T13:18:54.357907+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:54.357907+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "40'4500", 'ondisk_log_start': "40'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27238883, 'num_objects': 6651, 'num_object_clones': 0, 'num_object_copies': 13302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11210, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2086}, {'shards': '0,2', 'objects': 625}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:18:56.212 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:18:56.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:56.941 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45344/212378 objects degraded (21.351%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":463,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":106189,"data_bytes":434934286,"bytes_used":2112847872,"bytes_avail":287797444608,"bytes_total":289910292480,"degraded_objects":45344,"degraded_total":212378,"degraded_ratio":0.21350610703556869,"misplaced_objects":576,"misplaced_total":212378,"misplaced_ratio":0.0027121453257870406,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":131368,"recovering_keys_per_sec":0,"num_objects_recovered":385,"num_bytes_recovered":1576960,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":486},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:18:56.975 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:18:56.975 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:18:57.566 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:18:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:18:57.589 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-05T13:18:55.659711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":434934286,"num_objects":106189,"num_object_clones":0,"num_object_copies":212378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":576,"num_objects_unfound":0,"num_objects_dirty":106189,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185002,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7579,"num_bytes_recovered":31023123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160885,"ondisk_log_size":160885,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2063328,"kb_used_data":1039516,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281052192,"statfs":{"total":289910292480,"available":287797444608,"internally_reserved":0,"allocated":1064464384,"data_stored":1063616597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1048300550},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,5,8,15,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23941120,"num_objects":-5845,"num_object_clones":0,"num_object_copies":-11690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":61,"num_objects_misplaced":-446,"num_objects_unfound":0,"num_objects_dirty":-5845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5845,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":1576960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004091"},"pg_stats":[{"pgid":"2.f","version":"45'14280","reported_seq":14356,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.398679+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:54.398679+0000","last_peered":"2024-09-05T13:18:54.398679+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:54.398679+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410432,"num_objects":6692,"num_object_clones":0,"num_object_copies":13384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14280,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2271},{"shards":"0,2","objects":626}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14200","reported_seq":14305,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.413660+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:53.413660+0000","last_peered":"2024-09-05T13:18:53.413660+0000","last_clean":"2024-09-05T13:18:53.413660+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:53.413660+0000","last_undegraded":"2024-09-05T13:18:53.413660+0000","last_fullsized":"2024-09-05T13:18:53.413660+0000","mapping_epoch":35,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26845184,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12210,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14503","reported_seq":14630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.356957+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:53.356957+0000","last_peered":"2024-09-05T13:18:53.356957+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:53.356957+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27410456,"num_objects":6693,"num_object_clones":0,"num_object_copies":13386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12314,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":589},{"shards":"2","objects":2201}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14304","reported_seq":14374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.413669+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:53.413669+0000","last_peered":"2024-09-05T13:18:53.413669+0000","last_clean":"2024-09-05T13:18:53.413669+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:53.413669+0000","last_undegraded":"2024-09-05T13:18:53.413669+0000","last_fullsized":"2024-09-05T13:18:53.413669+0000","mapping_epoch":35,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27533312,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14304,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14275","reported_seq":14410,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:54.424605+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:54.424605+0000","last_peered":"2024-09-05T13:18:54.424605+0000","last_clean":"2024-09-05T13:18:54.424605+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:54.424605+0000","last_undegraded":"2024-09-05T13:18:54.424605+0000","last_fullsized":"2024-09-05T13:18:54.424605+0000","mapping_epoch":34,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27066368,"num_objects":6608,"num_object_clones":0,"num_object_copies":13216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10815,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14438","reported_seq":14576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.489303+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:54.489303+0000","last_peered":"2024-09-05T13:18:54.489303+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:54.489303+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27512832,"num_objects":6717,"num_object_clones":0,"num_object_copies":13434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10893,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2266},{"shards":"0,2","objects":593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14158","reported_seq":20672,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.472802+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:54.472802+0000","last_peered":"2024-09-05T13:18:54.472802+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:54.472802+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26501120,"num_objects":6470,"num_object_clones":0,"num_object_copies":12940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":576,"num_objects_unfound":0,"num_objects_dirty":6470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10477,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3186,"num_bytes_recovered":13041664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":576}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14430","reported_seq":14563,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:54.320738+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:54.320738+0000","last_peered":"2024-09-05T13:18:54.320738+0000","last_clean":"2024-09-05T13:18:54.320738+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:54.320738+0000","last_undegraded":"2024-09-05T13:18:54.320738+0000","last_fullsized":"2024-09-05T13:18:54.320738+0000","mapping_epoch":34,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27336704,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11040,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14383","reported_seq":14519,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.354314+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:53.354314+0000","last_peered":"2024-09-05T13:18:53.354314+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:53.354314+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27484160,"num_objects":6710,"num_object_clones":0,"num_object_copies":13420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12037,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":618},{"shards":"2","objects":2259}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14296","reported_seq":14434,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.435006+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:54.435006+0000","last_peered":"2024-09-05T13:18:54.435006+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:54.435006+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27090944,"num_objects":6614,"num_object_clones":0,"num_object_copies":13228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10663,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2160},{"shards":"0,2","objects":617}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14217","reported_seq":23119,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:53.433590+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:53.433590+0000","last_peered":"2024-09-05T13:18:53.433590+0000","last_clean":"2024-09-05T13:18:53.433590+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:53.433590+0000","last_undegraded":"2024-09-05T13:18:53.433590+0000","last_fullsized":"2024-09-05T13:18:53.433590+0000","mapping_epoch":44,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27418624,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11812,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14067","reported_seq":14191,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:53.317958+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:53.317958+0000","last_peered":"2024-09-05T13:18:53.317958+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:53.317958+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26685440,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11576,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":614},{"shards":"2","objects":2128}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14287","reported_seq":14426,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.418118+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:54.418118+0000","last_peered":"2024-09-05T13:18:54.418118+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:54.418118+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26996736,"num_objects":6591,"num_object_clones":0,"num_object_copies":13182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10438,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2172},{"shards":"0,2","objects":572}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14414","reported_seq":14602,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.458108+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:54.458108+0000","last_peered":"2024-09-05T13:18:54.458108+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:54.458108+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27402240,"num_objects":6690,"num_object_clones":0,"num_object_copies":13380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10313,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2246},{"shards":"0,2","objects":586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14488","reported_seq":14626,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.344586+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:54.344586+0000","last_peered":"2024-09-05T13:18:54.344586+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:54.344586+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27000832,"num_objects":6592,"num_object_clones":0,"num_object_copies":13184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10618,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026},{"shards":"0,2","objects":620}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14542","reported_seq":14683,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:54.357907+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:54.357907+0000","last_peered":"2024-09-05T13:18:54.357907+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:54.357907+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"40'4500","ondisk_log_start":"40'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27238883,"num_objects":6651,"num_object_clones":0,"num_object_copies":13302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11210,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2086},{"shards":"0,2","objects":625}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":434934267,"num_objects":106187,"num_object_clones":0,"num_object_copies":212374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":576,"num_objects_unfound":0,"num_objects_dirty":106187,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185000,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7578,"num_bytes_recovered":31023104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1061740544,"data_stored":1061656527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160882,"ondisk_log_size":160882,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663696,"kb_used_data":357104,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93708144,"statfs":{"total":96636764160,"available":95957139456,"internally_reserved":0,"allocated":365674496,"data_stored":365414939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":786640,"kb_used_data":389884,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93585200,"statfs":{"total":96636764160,"available":95831244800,"internally_reserved":0,"allocated":399241216,"data_stored":398981654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612992,"kb_used_data":292528,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93758848,"statfs":{"total":96636764160,"available":96009060352,"internally_reserved":0,"allocated":299548672,"data_stored":299220004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,4,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":398254080,"data_stored":398250467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299073536,"data_stored":299000817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364412928,"data_stored":364405243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:00.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:01.131 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:01.132 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:01.156 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-05T13:18:59.661139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":426340878,"num_objects":104091,"num_object_clones":0,"num_object_copies":208182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":104091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187100,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7716,"num_bytes_recovered":31584275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087188,"kb_used_data":1028700,"kb_used_omap":12,"kb_used_meta":1058419,"kb_avail":281028332,"statfs":{"total":289910292480,"available":287773011968,"internally_reserved":0,"allocated":1053388800,"data_stored":1052541013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1083821062},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,10,3,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26652672,"num_objects":-6507,"num_object_clones":0,"num_object_copies":-13014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":-438,"num_objects_unfound":0,"num_objects_dirty":-6507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6507,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":1638400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004226"},"pg_stats":[{"pgid":"2.f","version":"45'14417","reported_seq":14489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.449581+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:59.449581+0000","last_peered":"2024-09-05T13:18:59.449581+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:59.449581+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26849280,"num_objects":6555,"num_object_clones":0,"num_object_copies":13110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14417,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134},{"shards":"0,2","objects":489}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14298","reported_seq":14403,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.425575+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:58.425575+0000","last_peered":"2024-09-05T13:18:58.425575+0000","last_clean":"2024-09-05T13:18:58.425575+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:58.425575+0000","last_undegraded":"2024-09-05T13:18:58.425575+0000","last_fullsized":"2024-09-05T13:18:58.425575+0000","mapping_epoch":35,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26443776,"num_objects":6456,"num_object_clones":0,"num_object_copies":12912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12308,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14622","reported_seq":14749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.284282+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:58.284282+0000","last_peered":"2024-09-05T13:18:58.284282+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:58.284282+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923032,"num_objects":6574,"num_object_clones":0,"num_object_copies":13148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6574,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12433,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":470},{"shards":"2","objects":2082}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14441","reported_seq":14511,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.346906+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:58.346906+0000","last_peered":"2024-09-05T13:18:58.346906+0000","last_clean":"2024-09-05T13:18:58.346906+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:58.346906+0000","last_undegraded":"2024-09-05T13:18:58.346906+0000","last_fullsized":"2024-09-05T13:18:58.346906+0000","mapping_epoch":35,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26972160,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14441,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14424","reported_seq":14559,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:59.457212+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:59.457212+0000","last_peered":"2024-09-05T13:18:59.457212+0000","last_clean":"2024-09-05T13:18:59.457212+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:59.457212+0000","last_undegraded":"2024-09-05T13:18:59.457212+0000","last_fullsized":"2024-09-05T13:18:59.457212+0000","mapping_epoch":34,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26456064,"num_objects":6459,"num_object_clones":0,"num_object_copies":12918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10964,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14579","reported_seq":14718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.496392+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:59.496392+0000","last_peered":"2024-09-05T13:18:59.496392+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:59.496392+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11034,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2125},{"shards":"0,2","objects":452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14287","reported_seq":21074,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.506297+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:59.506297+0000","last_peered":"2024-09-05T13:18:59.506297+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:59.506297+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":13602816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5226},{"shards":"0,2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14553","reported_seq":14686,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:59.476770+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:59.476770+0000","last_peered":"2024-09-05T13:18:59.476770+0000","last_clean":"2024-09-05T13:18:59.476770+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:59.476770+0000","last_undegraded":"2024-09-05T13:18:59.476770+0000","last_fullsized":"2024-09-05T13:18:59.476770+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26832896,"num_objects":6551,"num_object_clones":0,"num_object_copies":13102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14537","reported_seq":14673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.370315+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:58.370315+0000","last_peered":"2024-09-05T13:18:58.370315+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:58.370315+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":464},{"shards":"2","objects":2105}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14436","reported_seq":14575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.519025+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:59.519025+0000","last_peered":"2024-09-05T13:18:59.519025+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:59.519025+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10803,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2020},{"shards":"0,2","objects":477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14333","reported_seq":23235,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.422571+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:58.422571+0000","last_peered":"2024-09-05T13:18:58.422571+0000","last_clean":"2024-09-05T13:18:58.422571+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:58.422571+0000","last_undegraded":"2024-09-05T13:18:58.422571+0000","last_fullsized":"2024-09-05T13:18:58.422571+0000","mapping_epoch":44,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26943488,"num_objects":6578,"num_object_clones":0,"num_object_copies":13156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11928,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14183","reported_seq":14307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.376458+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:58.376458+0000","last_peered":"2024-09-05T13:18:58.376458+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:58.376458+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26210304,"num_objects":6399,"num_object_clones":0,"num_object_copies":12798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":498},{"shards":"2","objects":2012}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14405","reported_seq":14541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.446580+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:59.446580+0000","last_peered":"2024-09-05T13:18:59.446580+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:59.446580+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10556,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2054},{"shards":"0,2","objects":454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14563","reported_seq":14751,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.414256+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:59.414256+0000","last_peered":"2024-09-05T13:18:59.414256+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:59.414256+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10462,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2097},{"shards":"0,2","objects":437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14626","reported_seq":14766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.466152+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:59.466152+0000","last_peered":"2024-09-05T13:18:59.466152+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:59.466152+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26435584,"num_objects":6454,"num_object_clones":0,"num_object_copies":12908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10756,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14676","reported_seq":14817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.364383+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:59.364383+0000","last_peered":"2024-09-05T13:18:59.364383+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:59.364383+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26690019,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11344,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1952},{"shards":"0,2","objects":491}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":426340859,"num_objects":104089,"num_object_clones":0,"num_object_copies":208178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":104089,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187098,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7715,"num_bytes_recovered":31584256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1050705920,"data_stored":1050621903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663776,"kb_used_data":353036,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93708064,"statfs":{"total":96636764160,"available":95957057536,"internally_reserved":0,"allocated":361508864,"data_stored":361249307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":802480,"kb_used_data":383520,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93569360,"statfs":{"total":96636764160,"available":95815024640,"internally_reserved":0,"allocated":392724480,"data_stored":392464918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620932,"kb_used_data":292144,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750908,"statfs":{"total":96636764160,"available":96000929792,"internally_reserved":0,"allocated":299155456,"data_stored":298826788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":391626752,"data_stored":391623139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298733568,"data_stored":298660849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360345600,"data_stored":360337915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:01.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:01.725 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:01.761 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-05T13:19:01.661850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":426340878,"num_objects":104091,"num_object_clones":0,"num_object_copies":208182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":104091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187100,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7716,"num_bytes_recovered":31584275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087188,"kb_used_data":1028700,"kb_used_omap":12,"kb_used_meta":1058419,"kb_avail":281028332,"statfs":{"total":289910292480,"available":287773011968,"internally_reserved":0,"allocated":1053388800,"data_stored":1052541013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1083821062},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,10,3,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17678336,"num_objects":-4316,"num_object_clones":0,"num_object_copies":-8632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":-4316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4316,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004295"},"pg_stats":[{"pgid":"2.f","version":"45'14417","reported_seq":14489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.449581+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:59.449581+0000","last_peered":"2024-09-05T13:18:59.449581+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:59.449581+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26849280,"num_objects":6555,"num_object_clones":0,"num_object_copies":13110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14417,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134},{"shards":"0,2","objects":489}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14298","reported_seq":14403,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.425575+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:58.425575+0000","last_peered":"2024-09-05T13:18:58.425575+0000","last_clean":"2024-09-05T13:18:58.425575+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:58.425575+0000","last_undegraded":"2024-09-05T13:18:58.425575+0000","last_fullsized":"2024-09-05T13:18:58.425575+0000","mapping_epoch":35,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26443776,"num_objects":6456,"num_object_clones":0,"num_object_copies":12912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12308,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14622","reported_seq":14749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.284282+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:58.284282+0000","last_peered":"2024-09-05T13:18:58.284282+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:58.284282+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923032,"num_objects":6574,"num_object_clones":0,"num_object_copies":13148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6574,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12433,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":470},{"shards":"2","objects":2082}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14441","reported_seq":14511,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.346906+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:58.346906+0000","last_peered":"2024-09-05T13:18:58.346906+0000","last_clean":"2024-09-05T13:18:58.346906+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:58.346906+0000","last_undegraded":"2024-09-05T13:18:58.346906+0000","last_fullsized":"2024-09-05T13:18:58.346906+0000","mapping_epoch":35,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26972160,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14441,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14424","reported_seq":14559,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:59.457212+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:59.457212+0000","last_peered":"2024-09-05T13:18:59.457212+0000","last_clean":"2024-09-05T13:18:59.457212+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:59.457212+0000","last_undegraded":"2024-09-05T13:18:59.457212+0000","last_fullsized":"2024-09-05T13:18:59.457212+0000","mapping_epoch":34,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26456064,"num_objects":6459,"num_object_clones":0,"num_object_copies":12918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10964,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14579","reported_seq":14718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.496392+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:59.496392+0000","last_peered":"2024-09-05T13:18:59.496392+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:59.496392+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11034,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2125},{"shards":"0,2","objects":452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14287","reported_seq":21074,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.506297+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:59.506297+0000","last_peered":"2024-09-05T13:18:59.506297+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:59.506297+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":13602816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5226},{"shards":"0,2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14553","reported_seq":14686,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:59.476770+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:59.476770+0000","last_peered":"2024-09-05T13:18:59.476770+0000","last_clean":"2024-09-05T13:18:59.476770+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:59.476770+0000","last_undegraded":"2024-09-05T13:18:59.476770+0000","last_fullsized":"2024-09-05T13:18:59.476770+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26832896,"num_objects":6551,"num_object_clones":0,"num_object_copies":13102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14537","reported_seq":14673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.370315+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:58.370315+0000","last_peered":"2024-09-05T13:18:58.370315+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:58.370315+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":464},{"shards":"2","objects":2105}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14436","reported_seq":14575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.519025+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:59.519025+0000","last_peered":"2024-09-05T13:18:59.519025+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:59.519025+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10803,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2020},{"shards":"0,2","objects":477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14333","reported_seq":23235,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.422571+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:58.422571+0000","last_peered":"2024-09-05T13:18:58.422571+0000","last_clean":"2024-09-05T13:18:58.422571+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:58.422571+0000","last_undegraded":"2024-09-05T13:18:58.422571+0000","last_fullsized":"2024-09-05T13:18:58.422571+0000","mapping_epoch":44,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26943488,"num_objects":6578,"num_object_clones":0,"num_object_copies":13156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11928,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14183","reported_seq":14307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.376458+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:58.376458+0000","last_peered":"2024-09-05T13:18:58.376458+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:58.376458+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26210304,"num_objects":6399,"num_object_clones":0,"num_object_copies":12798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":498},{"shards":"2","objects":2012}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14405","reported_seq":14541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.446580+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:59.446580+0000","last_peered":"2024-09-05T13:18:59.446580+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:59.446580+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10556,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2054},{"shards":"0,2","objects":454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14563","reported_seq":14751,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.414256+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:59.414256+0000","last_peered":"2024-09-05T13:18:59.414256+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:59.414256+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10462,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2097},{"shards":"0,2","objects":437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14626","reported_seq":14766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.466152+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:59.466152+0000","last_peered":"2024-09-05T13:18:59.466152+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:59.466152+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26435584,"num_objects":6454,"num_object_clones":0,"num_object_copies":12908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10756,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14676","reported_seq":14817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.364383+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:59.364383+0000","last_peered":"2024-09-05T13:18:59.364383+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:59.364383+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26690019,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11344,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1952},{"shards":"0,2","objects":491}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":426340859,"num_objects":104089,"num_object_clones":0,"num_object_copies":208178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":104089,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187098,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7715,"num_bytes_recovered":31584256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1050705920,"data_stored":1050621903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663776,"kb_used_data":353036,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93708064,"statfs":{"total":96636764160,"available":95957057536,"internally_reserved":0,"allocated":361508864,"data_stored":361249307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":802480,"kb_used_data":383520,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93569360,"statfs":{"total":96636764160,"available":95815024640,"internally_reserved":0,"allocated":392724480,"data_stored":392464918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620932,"kb_used_data":292144,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750908,"statfs":{"total":96636764160,"available":96000929792,"internally_reserved":0,"allocated":299155456,"data_stored":298826788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":391626752,"data_stored":391623139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298733568,"data_stored":298660849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360345600,"data_stored":360337915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:01.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:01.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'14417", 'reported_seq': 14489, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:59.449581+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:18:59.449581+0000', 'last_peered': '2024-09-05T13:18:59.449581+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:18:59.449581+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "40'4400", 'ondisk_log_start': "40'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26849280, 'num_objects': 6555, 'num_object_clones': 0, 'num_object_copies': 13110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14417, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2134}, {'shards': '0,2', 'objects': 489}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:01.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:01.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'14622", 'reported_seq': 14749, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:58.284282+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:18:58.284282+0000', 'last_peered': '2024-09-05T13:18:58.284282+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:18:58.284282+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26923032, 'num_objects': 6574, 'num_object_clones': 0, 'num_object_copies': 13148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6574, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12433, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 470}, {'shards': '2', 'objects': 2082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:01.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:01.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'14579", 'reported_seq': 14718, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:59.496392+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:18:59.496392+0000', 'last_peered': '2024-09-05T13:18:59.496392+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:18:59.496392+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26935296, 'num_objects': 6576, 'num_object_clones': 0, 'num_object_copies': 13152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11034, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2125}, {'shards': '0,2', 'objects': 452}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:01.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:19:01.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'14287", 'reported_seq': 21074, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:59.506297+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:18:59.506297+0000', 'last_peered': '2024-09-05T13:18:59.506297+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:18:59.506297+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25972736, 'num_objects': 6341, 'num_object_clones': 0, 'num_object_copies': 12682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 676, 'num_objects_misplaced': 439, 'num_objects_unfound': 0, 'num_objects_dirty': 6341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10606, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3323, 'num_bytes_recovered': 13602816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5226}, {'shards': '0,2', 'objects': 439}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:01.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:01.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'14537", 'reported_seq': 14673, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:58.370315+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:18:58.370315+0000', 'last_peered': '2024-09-05T13:18:58.370315+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:18:58.370315+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26853376, 'num_objects': 6556, 'num_object_clones': 0, 'num_object_copies': 13112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6556, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12191, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 464}, {'shards': '2', 'objects': 2105}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:01.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:01.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'14436", 'reported_seq': 14575, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:59.519025+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:18:59.519025+0000', 'last_peered': '2024-09-05T13:18:59.519025+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:18:59.519025+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "40'4400", 'ondisk_log_start': "40'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26517504, 'num_objects': 6474, 'num_object_clones': 0, 'num_object_copies': 12948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10803, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2020}, {'shards': '0,2', 'objects': 477}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:01.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:01.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'14183", 'reported_seq': 14307, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:58.376458+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:18:58.376458+0000', 'last_peered': '2024-09-05T13:18:58.376458+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:18:58.376458+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26210304, 'num_objects': 6399, 'num_object_clones': 0, 'num_object_copies': 12798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6399, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11692, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 498}, {'shards': '2', 'objects': 2012}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:01.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:01.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'14405", 'reported_seq': 14541, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:59.446580+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:18:59.446580+0000', 'last_peered': '2024-09-05T13:18:59.446580+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:18:59.446580+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "40'4400", 'ondisk_log_start': "40'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26513408, 'num_objects': 6473, 'num_object_clones': 0, 'num_object_copies': 12946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10556, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2054}, {'shards': '0,2', 'objects': 454}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:01.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:01.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'14563", 'reported_seq': 14751, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:59.414256+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:18:59.414256+0000', 'last_peered': '2024-09-05T13:18:59.414256+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:18:59.414256+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26791936, 'num_objects': 6541, 'num_object_clones': 0, 'num_object_copies': 13082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10462, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2097}, {'shards': '0,2', 'objects': 437}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:01.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:01.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'14626", 'reported_seq': 14766, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:59.466152+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:18:59.466152+0000', 'last_peered': '2024-09-05T13:18:59.466152+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:18:59.466152+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26435584, 'num_objects': 6454, 'num_object_clones': 0, 'num_object_copies': 12908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6454, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10756, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1888}, {'shards': '0,2', 'objects': 482}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:01.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:01.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'14676", 'reported_seq': 14817, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:18:59.364383+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:18:59.364383+0000', 'last_peered': '2024-09-05T13:18:59.364383+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:18:59.364383+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26690019, 'num_objects': 6517, 'num_object_clones': 0, 'num_object_copies': 13034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11344, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1952}, {'shards': '0,2', 'objects': 491}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:01.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:02.461 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:02.461 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45344/208182 objects degraded (21.781%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":469,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":104091,"data_bytes":426340878,"bytes_used":2137280512,"bytes_avail":287773011968,"bytes_total":289910292480,"degraded_objects":45344,"degraded_total":208182,"degraded_ratio":0.21780941676033472,"misplaced_objects":439,"misplaced_total":208182,"misplaced_ratio":0.0021087317827670018,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93150,"recovering_keys_per_sec":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":359},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:02.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:02.484 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:03.056 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:03.079 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-05T13:19:01.661850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":426340878,"num_objects":104091,"num_object_clones":0,"num_object_copies":208182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":104091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187100,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7716,"num_bytes_recovered":31584275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087188,"kb_used_data":1028700,"kb_used_omap":12,"kb_used_meta":1058419,"kb_avail":281028332,"statfs":{"total":289910292480,"available":287773011968,"internally_reserved":0,"allocated":1053388800,"data_stored":1052541013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1083821062},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,10,3,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17678336,"num_objects":-4316,"num_object_clones":0,"num_object_copies":-8632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":-4316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4316,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004295"},"pg_stats":[{"pgid":"2.f","version":"45'14417","reported_seq":14489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.449581+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:18:59.449581+0000","last_peered":"2024-09-05T13:18:59.449581+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:18:59.449581+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26849280,"num_objects":6555,"num_object_clones":0,"num_object_copies":13110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14417,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134},{"shards":"0,2","objects":489}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14298","reported_seq":14403,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.425575+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:18:58.425575+0000","last_peered":"2024-09-05T13:18:58.425575+0000","last_clean":"2024-09-05T13:18:58.425575+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:18:58.425575+0000","last_undegraded":"2024-09-05T13:18:58.425575+0000","last_fullsized":"2024-09-05T13:18:58.425575+0000","mapping_epoch":35,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26443776,"num_objects":6456,"num_object_clones":0,"num_object_copies":12912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12308,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14622","reported_seq":14749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.284282+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:18:58.284282+0000","last_peered":"2024-09-05T13:18:58.284282+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:18:58.284282+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26923032,"num_objects":6574,"num_object_clones":0,"num_object_copies":13148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6574,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12433,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":470},{"shards":"2","objects":2082}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14441","reported_seq":14511,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.346906+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:18:58.346906+0000","last_peered":"2024-09-05T13:18:58.346906+0000","last_clean":"2024-09-05T13:18:58.346906+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:18:58.346906+0000","last_undegraded":"2024-09-05T13:18:58.346906+0000","last_fullsized":"2024-09-05T13:18:58.346906+0000","mapping_epoch":35,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26972160,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14441,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14424","reported_seq":14559,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:59.457212+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:18:59.457212+0000","last_peered":"2024-09-05T13:18:59.457212+0000","last_clean":"2024-09-05T13:18:59.457212+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:18:59.457212+0000","last_undegraded":"2024-09-05T13:18:59.457212+0000","last_fullsized":"2024-09-05T13:18:59.457212+0000","mapping_epoch":34,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26456064,"num_objects":6459,"num_object_clones":0,"num_object_copies":12918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10964,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14579","reported_seq":14718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.496392+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:18:59.496392+0000","last_peered":"2024-09-05T13:18:59.496392+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:18:59.496392+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26935296,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11034,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2125},{"shards":"0,2","objects":452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14287","reported_seq":21074,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.506297+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:18:59.506297+0000","last_peered":"2024-09-05T13:18:59.506297+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:18:59.506297+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":13602816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5226},{"shards":"0,2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14553","reported_seq":14686,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:59.476770+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:18:59.476770+0000","last_peered":"2024-09-05T13:18:59.476770+0000","last_clean":"2024-09-05T13:18:59.476770+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:18:59.476770+0000","last_undegraded":"2024-09-05T13:18:59.476770+0000","last_fullsized":"2024-09-05T13:18:59.476770+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26832896,"num_objects":6551,"num_object_clones":0,"num_object_copies":13102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14537","reported_seq":14673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.370315+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:18:58.370315+0000","last_peered":"2024-09-05T13:18:58.370315+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:18:58.370315+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12191,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":464},{"shards":"2","objects":2105}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14436","reported_seq":14575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.519025+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:18:59.519025+0000","last_peered":"2024-09-05T13:18:59.519025+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:18:59.519025+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10803,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2020},{"shards":"0,2","objects":477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14333","reported_seq":23235,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:18:58.422571+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:18:58.422571+0000","last_peered":"2024-09-05T13:18:58.422571+0000","last_clean":"2024-09-05T13:18:58.422571+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:18:58.422571+0000","last_undegraded":"2024-09-05T13:18:58.422571+0000","last_fullsized":"2024-09-05T13:18:58.422571+0000","mapping_epoch":44,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26943488,"num_objects":6578,"num_object_clones":0,"num_object_copies":13156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11928,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14183","reported_seq":14307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:58.376458+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:18:58.376458+0000","last_peered":"2024-09-05T13:18:58.376458+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:18:58.376458+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26210304,"num_objects":6399,"num_object_clones":0,"num_object_copies":12798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":498},{"shards":"2","objects":2012}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14405","reported_seq":14541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.446580+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:18:59.446580+0000","last_peered":"2024-09-05T13:18:59.446580+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:18:59.446580+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10556,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2054},{"shards":"0,2","objects":454}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14563","reported_seq":14751,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.414256+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:18:59.414256+0000","last_peered":"2024-09-05T13:18:59.414256+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:18:59.414256+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10462,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2097},{"shards":"0,2","objects":437}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14626","reported_seq":14766,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.466152+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:18:59.466152+0000","last_peered":"2024-09-05T13:18:59.466152+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:18:59.466152+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26435584,"num_objects":6454,"num_object_clones":0,"num_object_copies":12908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10756,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14676","reported_seq":14817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:18:59.364383+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:18:59.364383+0000","last_peered":"2024-09-05T13:18:59.364383+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:18:59.364383+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26690019,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11344,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1952},{"shards":"0,2","objects":491}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":426340859,"num_objects":104089,"num_object_clones":0,"num_object_copies":208178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":439,"num_objects_unfound":0,"num_objects_dirty":104089,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187098,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7715,"num_bytes_recovered":31584256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1050705920,"data_stored":1050621903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663776,"kb_used_data":353036,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93708064,"statfs":{"total":96636764160,"available":95957057536,"internally_reserved":0,"allocated":361508864,"data_stored":361249307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":802480,"kb_used_data":383520,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93569360,"statfs":{"total":96636764160,"available":95815024640,"internally_reserved":0,"allocated":392724480,"data_stored":392464918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620932,"kb_used_data":292144,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750908,"statfs":{"total":96636764160,"available":96000929792,"internally_reserved":0,"allocated":299155456,"data_stored":298826788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":391626752,"data_stored":391623139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298733568,"data_stored":298660849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360345600,"data_stored":360337915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:06.082 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:06.675 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:06.677 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:06.696 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-05T13:19:05.662979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":417096206,"num_objects":101834,"num_object_clones":0,"num_object_copies":203668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":101834,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189357,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7850,"num_bytes_recovered":32133139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2037832,"kb_used_data":1016792,"kb_used_omap":12,"kb_used_meta":1020979,"kb_avail":281077688,"statfs":{"total":289910292480,"available":287823552512,"internally_reserved":0,"allocated":1041195008,"data_stored":1040347221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1045482502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,9,4,9,12,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23445504,"num_objects":-5724,"num_object_clones":0,"num_object_copies":-11448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":-424,"num_objects_unfound":0,"num_objects_dirty":-5724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5724,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":1667072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004128"},"pg_stats":[{"pgid":"2.f","version":"45'14575","reported_seq":14650,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.549244+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:04.549244+0000","last_peered":"2024-09-05T13:19:04.549244+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:04.549244+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14575,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1976},{"shards":"0,2","objects":331}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14425","reported_seq":14531,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.470506+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:03.470506+0000","last_peered":"2024-09-05T13:19:03.470506+0000","last_clean":"2024-09-05T13:19:03.470506+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:03.470506+0000","last_undegraded":"2024-09-05T13:19:03.470506+0000","last_fullsized":"2024-09-05T13:19:03.470506+0000","mapping_epoch":35,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25923584,"num_objects":6329,"num_object_clones":0,"num_object_copies":12658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12435,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14744","reported_seq":14873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.448664+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:03.448664+0000","last_peered":"2024-09-05T13:19:03.448664+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:03.448664+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26423320,"num_objects":6452,"num_object_clones":0,"num_object_copies":12904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12555,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":348},{"shards":"2","objects":1960}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14602","reported_seq":14673,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.460222+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:03.460222+0000","last_peered":"2024-09-05T13:19:03.460222+0000","last_clean":"2024-09-05T13:19:03.460222+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:03.460222+0000","last_undegraded":"2024-09-05T13:19:03.460222+0000","last_fullsized":"2024-09-05T13:19:03.460222+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26312704,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14602,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14573","reported_seq":14708,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:04.522573+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:04.522573+0000","last_peered":"2024-09-05T13:19:04.522573+0000","last_clean":"2024-09-05T13:19:04.522573+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:04.522573+0000","last_undegraded":"2024-09-05T13:19:04.522573+0000","last_fullsized":"2024-09-05T13:19:04.522573+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25845760,"num_objects":6310,"num_object_clones":0,"num_object_copies":12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11113,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14708","reported_seq":14846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.519019+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:04.519019+0000","last_peered":"2024-09-05T13:19:04.519019+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:04.519019+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1996},{"shards":"0,2","objects":323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14429","reported_seq":21485,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.551860+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:04.551860+0000","last_peered":"2024-09-05T13:19:04.551860+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:04.551860+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25391104,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10748,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3457,"num_bytes_recovered":14151680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":305}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14699","reported_seq":14832,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:04.511494+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:04.511494+0000","last_peered":"2024-09-05T13:19:04.511494+0000","last_clean":"2024-09-05T13:19:04.511494+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:04.511494+0000","last_undegraded":"2024-09-05T13:19:04.511494+0000","last_fullsized":"2024-09-05T13:19:04.511494+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26234880,"num_objects":6405,"num_object_clones":0,"num_object_copies":12810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11309,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14659","reported_seq":14795,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.404359+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:03.404359+0000","last_peered":"2024-09-05T13:19:03.404359+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:03.404359+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353664,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12313,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":342},{"shards":"2","objects":1983}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14559","reported_seq":14698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.473901+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:04.473901+0000","last_peered":"2024-09-05T13:19:04.473901+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:04.473901+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26013696,"num_objects":6351,"num_object_clones":0,"num_object_copies":12702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10926,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1897},{"shards":"0,2","objects":354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14481","reported_seq":23383,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.389283+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:03.389283+0000","last_peered":"2024-09-05T13:19:03.389283+0000","last_clean":"2024-09-05T13:19:03.389283+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:03.389283+0000","last_undegraded":"2024-09-05T13:19:03.389283+0000","last_fullsized":"2024-09-05T13:19:03.389283+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337280,"num_objects":6430,"num_object_clones":0,"num_object_copies":12860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12076,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14335","reported_seq":14459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.406016+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:03.406016+0000","last_peered":"2024-09-05T13:19:03.406016+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:03.406016+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25587712,"num_objects":6247,"num_object_clones":0,"num_object_copies":12494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11844,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":346},{"shards":"2","objects":1860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14543","reported_seq":14684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.475919+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:04.475919+0000","last_peered":"2024-09-05T13:19:04.475919+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:04.475919+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25948160,"num_objects":6335,"num_object_clones":0,"num_object_copies":12670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10694,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1916},{"shards":"0,2","objects":316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14703","reported_seq":14891,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.524766+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:04.524766+0000","last_peered":"2024-09-05T13:19:04.524766+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:04.524766+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218496,"num_objects":6401,"num_object_clones":0,"num_object_copies":12802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1957},{"shards":"0,2","objects":297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14770","reported_seq":14910,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.548675+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:04.548675+0000","last_peered":"2024-09-05T13:19:04.548675+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:04.548675+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25845760,"num_objects":6310,"num_object_clones":0,"num_object_copies":12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10900,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1744},{"shards":"0,2","objects":338}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14832","reported_seq":14973,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.503595+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:04.503595+0000","last_peered":"2024-09-05T13:19:04.503595+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:04.503595+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26051043,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11500,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1796},{"shards":"0,2","objects":335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":417096187,"num_objects":101832,"num_object_clones":0,"num_object_copies":203664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":101832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189355,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7849,"num_bytes_recovered":32133120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1038356480,"data_stored":1038272463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646816,"kb_used_data":348560,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93725024,"statfs":{"total":96636764160,"available":95974424576,"internally_reserved":0,"allocated":356925440,"data_stored":356665883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,3,3,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783204,"kb_used_data":376724,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93588636,"statfs":{"total":96636764160,"available":95834763264,"internally_reserved":0,"allocated":385765376,"data_stored":385505814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607812,"kb_used_data":291508,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93764028,"statfs":{"total":96636764160,"available":96014364672,"internally_reserved":0,"allocated":298504192,"data_stored":298175524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384638976,"data_stored":384635363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297992192,"data_stored":297919473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355725312,"data_stored":355717627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:06.698 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:07.245 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:07.272 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-05T13:19:05.662979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":417096206,"num_objects":101834,"num_object_clones":0,"num_object_copies":203668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":101834,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189357,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7850,"num_bytes_recovered":32133139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2037832,"kb_used_data":1016792,"kb_used_omap":12,"kb_used_meta":1020979,"kb_avail":281077688,"statfs":{"total":289910292480,"available":287823552512,"internally_reserved":0,"allocated":1041195008,"data_stored":1040347221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1045482502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,9,4,9,12,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23445504,"num_objects":-5724,"num_object_clones":0,"num_object_copies":-11448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":-424,"num_objects_unfound":0,"num_objects_dirty":-5724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5724,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":1667072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004128"},"pg_stats":[{"pgid":"2.f","version":"45'14575","reported_seq":14650,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.549244+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:04.549244+0000","last_peered":"2024-09-05T13:19:04.549244+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:04.549244+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14575,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1976},{"shards":"0,2","objects":331}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14425","reported_seq":14531,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.470506+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:03.470506+0000","last_peered":"2024-09-05T13:19:03.470506+0000","last_clean":"2024-09-05T13:19:03.470506+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:03.470506+0000","last_undegraded":"2024-09-05T13:19:03.470506+0000","last_fullsized":"2024-09-05T13:19:03.470506+0000","mapping_epoch":35,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25923584,"num_objects":6329,"num_object_clones":0,"num_object_copies":12658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12435,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14744","reported_seq":14873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.448664+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:03.448664+0000","last_peered":"2024-09-05T13:19:03.448664+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:03.448664+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26423320,"num_objects":6452,"num_object_clones":0,"num_object_copies":12904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12555,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":348},{"shards":"2","objects":1960}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14602","reported_seq":14673,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.460222+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:03.460222+0000","last_peered":"2024-09-05T13:19:03.460222+0000","last_clean":"2024-09-05T13:19:03.460222+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:03.460222+0000","last_undegraded":"2024-09-05T13:19:03.460222+0000","last_fullsized":"2024-09-05T13:19:03.460222+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26312704,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14602,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14573","reported_seq":14708,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:04.522573+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:04.522573+0000","last_peered":"2024-09-05T13:19:04.522573+0000","last_clean":"2024-09-05T13:19:04.522573+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:04.522573+0000","last_undegraded":"2024-09-05T13:19:04.522573+0000","last_fullsized":"2024-09-05T13:19:04.522573+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25845760,"num_objects":6310,"num_object_clones":0,"num_object_copies":12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11113,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14708","reported_seq":14846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.519019+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:04.519019+0000","last_peered":"2024-09-05T13:19:04.519019+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:04.519019+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1996},{"shards":"0,2","objects":323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14429","reported_seq":21485,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.551860+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:04.551860+0000","last_peered":"2024-09-05T13:19:04.551860+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:04.551860+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25391104,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10748,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3457,"num_bytes_recovered":14151680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":305}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14699","reported_seq":14832,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:04.511494+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:04.511494+0000","last_peered":"2024-09-05T13:19:04.511494+0000","last_clean":"2024-09-05T13:19:04.511494+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:04.511494+0000","last_undegraded":"2024-09-05T13:19:04.511494+0000","last_fullsized":"2024-09-05T13:19:04.511494+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26234880,"num_objects":6405,"num_object_clones":0,"num_object_copies":12810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11309,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14659","reported_seq":14795,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.404359+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:03.404359+0000","last_peered":"2024-09-05T13:19:03.404359+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:03.404359+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353664,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12313,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":342},{"shards":"2","objects":1983}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14559","reported_seq":14698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.473901+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:04.473901+0000","last_peered":"2024-09-05T13:19:04.473901+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:04.473901+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26013696,"num_objects":6351,"num_object_clones":0,"num_object_copies":12702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10926,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1897},{"shards":"0,2","objects":354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14481","reported_seq":23383,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.389283+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:03.389283+0000","last_peered":"2024-09-05T13:19:03.389283+0000","last_clean":"2024-09-05T13:19:03.389283+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:03.389283+0000","last_undegraded":"2024-09-05T13:19:03.389283+0000","last_fullsized":"2024-09-05T13:19:03.389283+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337280,"num_objects":6430,"num_object_clones":0,"num_object_copies":12860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12076,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14335","reported_seq":14459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.406016+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:03.406016+0000","last_peered":"2024-09-05T13:19:03.406016+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:03.406016+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25587712,"num_objects":6247,"num_object_clones":0,"num_object_copies":12494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11844,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":346},{"shards":"2","objects":1860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14543","reported_seq":14684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.475919+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:04.475919+0000","last_peered":"2024-09-05T13:19:04.475919+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:04.475919+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25948160,"num_objects":6335,"num_object_clones":0,"num_object_copies":12670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10694,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1916},{"shards":"0,2","objects":316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14703","reported_seq":14891,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.524766+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:04.524766+0000","last_peered":"2024-09-05T13:19:04.524766+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:04.524766+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218496,"num_objects":6401,"num_object_clones":0,"num_object_copies":12802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1957},{"shards":"0,2","objects":297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14770","reported_seq":14910,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.548675+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:04.548675+0000","last_peered":"2024-09-05T13:19:04.548675+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:04.548675+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25845760,"num_objects":6310,"num_object_clones":0,"num_object_copies":12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10900,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1744},{"shards":"0,2","objects":338}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14832","reported_seq":14973,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.503595+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:04.503595+0000","last_peered":"2024-09-05T13:19:04.503595+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:04.503595+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26051043,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11500,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1796},{"shards":"0,2","objects":335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":417096187,"num_objects":101832,"num_object_clones":0,"num_object_copies":203664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":101832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189355,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7849,"num_bytes_recovered":32133120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1038356480,"data_stored":1038272463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646816,"kb_used_data":348560,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93725024,"statfs":{"total":96636764160,"available":95974424576,"internally_reserved":0,"allocated":356925440,"data_stored":356665883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,3,3,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783204,"kb_used_data":376724,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93588636,"statfs":{"total":96636764160,"available":95834763264,"internally_reserved":0,"allocated":385765376,"data_stored":385505814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607812,"kb_used_data":291508,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93764028,"statfs":{"total":96636764160,"available":96014364672,"internally_reserved":0,"allocated":298504192,"data_stored":298175524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384638976,"data_stored":384635363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297992192,"data_stored":297919473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355725312,"data_stored":355717627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:07.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:07.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'14575", 'reported_seq': 14650, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:04.549244+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:04.549244+0000', 'last_peered': '2024-09-05T13:19:04.549244+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:04.549244+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26202112, 'num_objects': 6397, 'num_object_clones': 0, 'num_object_copies': 12794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6397, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14575, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1976}, {'shards': '0,2', 'objects': 331}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:07.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:07.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'14744", 'reported_seq': 14873, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:03.448664+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:19:03.448664+0000', 'last_peered': '2024-09-05T13:19:03.448664+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:03.448664+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26423320, 'num_objects': 6452, 'num_object_clones': 0, 'num_object_copies': 12904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6452, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12555, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 348}, {'shards': '2', 'objects': 1960}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'14708", 'reported_seq': 14846, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:04.519019+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:04.519019+0000', 'last_peered': '2024-09-05T13:19:04.519019+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:04.519019+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26406912, 'num_objects': 6447, 'num_object_clones': 0, 'num_object_copies': 12894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11163, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1996}, {'shards': '0,2', 'objects': 323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'14429", 'reported_seq': 21485, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:04.551860+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:19:04.551860+0000', 'last_peered': '2024-09-05T13:19:04.551860+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:19:04.551860+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "40'4400", 'ondisk_log_start': "40'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25391104, 'num_objects': 6199, 'num_object_clones': 0, 'num_object_copies': 12398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 676, 'num_objects_misplaced': 305, 'num_objects_unfound': 0, 'num_objects_dirty': 6199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10748, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3457, 'num_bytes_recovered': 14151680, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5218}, {'shards': '0,2', 'objects': 305}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'14659", 'reported_seq': 14795, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:03.404359+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:03.404359+0000', 'last_peered': '2024-09-05T13:19:03.404359+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:03.404359+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26353664, 'num_objects': 6434, 'num_object_clones': 0, 'num_object_copies': 12868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6434, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12313, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 342}, {'shards': '2', 'objects': 1983}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'14559", 'reported_seq': 14698, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:04.473901+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:04.473901+0000', 'last_peered': '2024-09-05T13:19:04.473901+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:04.473901+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26013696, 'num_objects': 6351, 'num_object_clones': 0, 'num_object_copies': 12702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6351, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10926, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1897}, {'shards': '0,2', 'objects': 354}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:07.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'14335", 'reported_seq': 14459, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:03.406016+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:03.406016+0000', 'last_peered': '2024-09-05T13:19:03.406016+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:03.406016+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "40'4300", 'ondisk_log_start': "40'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25587712, 'num_objects': 6247, 'num_object_clones': 0, 'num_object_copies': 12494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11844, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 346}, {'shards': '2', 'objects': 1860}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'14543", 'reported_seq': 14684, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:04.475919+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:04.475919+0000', 'last_peered': '2024-09-05T13:19:04.475919+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:04.475919+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25948160, 'num_objects': 6335, 'num_object_clones': 0, 'num_object_copies': 12670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10694, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1916}, {'shards': '0,2', 'objects': 316}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'14703", 'reported_seq': 14891, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:04.524766+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:04.524766+0000', 'last_peered': '2024-09-05T13:19:04.524766+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:04.524766+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26218496, 'num_objects': 6401, 'num_object_clones': 0, 'num_object_copies': 12802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10602, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1957}, {'shards': '0,2', 'objects': 297}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'14770", 'reported_seq': 14910, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:04.548675+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:04.548675+0000', 'last_peered': '2024-09-05T13:19:04.548675+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:04.548675+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25845760, 'num_objects': 6310, 'num_object_clones': 0, 'num_object_copies': 12620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10900, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1744}, {'shards': '0,2', 'objects': 338}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:07.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'14832", 'reported_seq': 14973, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:04.503595+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:04.503595+0000', 'last_peered': '2024-09-05T13:19:04.503595+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:04.503595+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26051043, 'num_objects': 6361, 'num_object_clones': 0, 'num_object_copies': 12722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6361, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11500, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1796}, {'shards': '0,2', 'objects': 335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:07.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:07.972 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:07.972 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45344/203668 objects degraded (22.264%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":474,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":101834,"data_bytes":417096206,"bytes_used":2086739968,"bytes_avail":287823552512,"bytes_total":289910292480,"degraded_objects":45344,"degraded_total":203668,"degraded_ratio":0.22263684034801737,"misplaced_objects":305,"misplaced_total":203668,"misplaced_ratio":0.0014975352043521809,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":92453,"recovering_keys_per_sec":0,"num_objects_recovered":271,"num_bytes_recovered":1110016,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":362},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:07.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:07.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:08.538 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:08.539 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:08.561 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-05T13:19:07.665882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":417096206,"num_objects":101834,"num_object_clones":0,"num_object_copies":203668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":101834,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189357,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7850,"num_bytes_recovered":32133139,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2037832,"kb_used_data":1016792,"kb_used_omap":12,"kb_used_meta":1020979,"kb_avail":281077688,"statfs":{"total":289910292480,"available":287823552512,"internally_reserved":0,"allocated":1041195008,"data_stored":1040347221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1045482502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,9,4,9,12,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17838080,"num_objects":-4355,"num_object_clones":0,"num_object_copies":-8710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-4355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4355,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":1110016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006171"},"pg_stats":[{"pgid":"2.f","version":"45'14575","reported_seq":14650,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.549244+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:04.549244+0000","last_peered":"2024-09-05T13:19:04.549244+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:04.549244+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26202112,"num_objects":6397,"num_object_clones":0,"num_object_copies":12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14575,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1976},{"shards":"0,2","objects":331}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14425","reported_seq":14531,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.470506+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:03.470506+0000","last_peered":"2024-09-05T13:19:03.470506+0000","last_clean":"2024-09-05T13:19:03.470506+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:03.470506+0000","last_undegraded":"2024-09-05T13:19:03.470506+0000","last_fullsized":"2024-09-05T13:19:03.470506+0000","mapping_epoch":35,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25923584,"num_objects":6329,"num_object_clones":0,"num_object_copies":12658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12435,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14744","reported_seq":14873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.448664+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:03.448664+0000","last_peered":"2024-09-05T13:19:03.448664+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:03.448664+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26423320,"num_objects":6452,"num_object_clones":0,"num_object_copies":12904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12555,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":348},{"shards":"2","objects":1960}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14602","reported_seq":14673,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.460222+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:03.460222+0000","last_peered":"2024-09-05T13:19:03.460222+0000","last_clean":"2024-09-05T13:19:03.460222+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:03.460222+0000","last_undegraded":"2024-09-05T13:19:03.460222+0000","last_fullsized":"2024-09-05T13:19:03.460222+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26312704,"num_objects":6424,"num_object_clones":0,"num_object_copies":12848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14602,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14573","reported_seq":14708,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:04.522573+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:04.522573+0000","last_peered":"2024-09-05T13:19:04.522573+0000","last_clean":"2024-09-05T13:19:04.522573+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:04.522573+0000","last_undegraded":"2024-09-05T13:19:04.522573+0000","last_fullsized":"2024-09-05T13:19:04.522573+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25845760,"num_objects":6310,"num_object_clones":0,"num_object_copies":12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11113,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14708","reported_seq":14846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.519019+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:04.519019+0000","last_peered":"2024-09-05T13:19:04.519019+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:04.519019+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11163,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1996},{"shards":"0,2","objects":323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14429","reported_seq":21485,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.551860+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:04.551860+0000","last_peered":"2024-09-05T13:19:04.551860+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:04.551860+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"40'4400","ondisk_log_start":"40'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25391104,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10748,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3457,"num_bytes_recovered":14151680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5218},{"shards":"0,2","objects":305}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14699","reported_seq":14832,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:04.511494+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:04.511494+0000","last_peered":"2024-09-05T13:19:04.511494+0000","last_clean":"2024-09-05T13:19:04.511494+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:04.511494+0000","last_undegraded":"2024-09-05T13:19:04.511494+0000","last_fullsized":"2024-09-05T13:19:04.511494+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26234880,"num_objects":6405,"num_object_clones":0,"num_object_copies":12810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11309,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14659","reported_seq":14795,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.404359+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:03.404359+0000","last_peered":"2024-09-05T13:19:03.404359+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:03.404359+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353664,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12313,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":342},{"shards":"2","objects":1983}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14559","reported_seq":14698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.473901+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:04.473901+0000","last_peered":"2024-09-05T13:19:04.473901+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:04.473901+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26013696,"num_objects":6351,"num_object_clones":0,"num_object_copies":12702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10926,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1897},{"shards":"0,2","objects":354}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14481","reported_seq":23383,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:03.389283+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:03.389283+0000","last_peered":"2024-09-05T13:19:03.389283+0000","last_clean":"2024-09-05T13:19:03.389283+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:03.389283+0000","last_undegraded":"2024-09-05T13:19:03.389283+0000","last_fullsized":"2024-09-05T13:19:03.389283+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337280,"num_objects":6430,"num_object_clones":0,"num_object_copies":12860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12076,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14335","reported_seq":14459,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:03.406016+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:03.406016+0000","last_peered":"2024-09-05T13:19:03.406016+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:03.406016+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"40'4300","ondisk_log_start":"40'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25587712,"num_objects":6247,"num_object_clones":0,"num_object_copies":12494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11844,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":346},{"shards":"2","objects":1860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14543","reported_seq":14684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.475919+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:04.475919+0000","last_peered":"2024-09-05T13:19:04.475919+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:04.475919+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25948160,"num_objects":6335,"num_object_clones":0,"num_object_copies":12670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10694,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1916},{"shards":"0,2","objects":316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14703","reported_seq":14891,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.524766+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:04.524766+0000","last_peered":"2024-09-05T13:19:04.524766+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:04.524766+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26218496,"num_objects":6401,"num_object_clones":0,"num_object_copies":12802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1957},{"shards":"0,2","objects":297}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14770","reported_seq":14910,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.548675+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:04.548675+0000","last_peered":"2024-09-05T13:19:04.548675+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:04.548675+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25845760,"num_objects":6310,"num_object_clones":0,"num_object_copies":12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10900,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1744},{"shards":"0,2","objects":338}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14832","reported_seq":14973,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:04.503595+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:04.503595+0000","last_peered":"2024-09-05T13:19:04.503595+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:04.503595+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26051043,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11500,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1796},{"shards":"0,2","objects":335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":417096187,"num_objects":101832,"num_object_clones":0,"num_object_copies":203664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":305,"num_objects_unfound":0,"num_objects_dirty":101832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":189355,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7849,"num_bytes_recovered":32133120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1038356480,"data_stored":1038272463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646816,"kb_used_data":348560,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93725024,"statfs":{"total":96636764160,"available":95974424576,"internally_reserved":0,"allocated":356925440,"data_stored":356665883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,3,3,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783204,"kb_used_data":376724,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93588636,"statfs":{"total":96636764160,"available":95834763264,"internally_reserved":0,"allocated":385765376,"data_stored":385505814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607812,"kb_used_data":291508,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93764028,"statfs":{"total":96636764160,"available":96014364672,"internally_reserved":0,"allocated":298504192,"data_stored":298175524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384638976,"data_stored":384635363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297992192,"data_stored":297919473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355725312,"data_stored":355717627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:11.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:12.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:12.165 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-05T13:19:11.667213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":407945742,"num_objects":99600,"num_object_clones":0,"num_object_copies":199200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":99600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191591,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8003,"num_bytes_recovered":32759827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046948,"kb_used_data":1005104,"kb_used_omap":12,"kb_used_meta":1041779,"kb_avail":281068572,"statfs":{"total":289910292480,"available":287814217728,"internally_reserved":0,"allocated":1029226496,"data_stored":1028378709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1066781702},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,4,15,5,14,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18395136,"num_objects":-4491,"num_object_clones":0,"num_object_copies":-8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":-4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4491,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006074"},"pg_stats":[{"pgid":"2.f","version":"45'14718","reported_seq":14794,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.554063+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:09.554063+0000","last_peered":"2024-09-05T13:19:09.554063+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:09.554063+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25616384,"num_objects":6254,"num_object_clones":0,"num_object_copies":12508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833},{"shards":"0,2","objects":188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14580","reported_seq":14685,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:08.430080+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:08.430080+0000","last_peered":"2024-09-05T13:19:08.430080+0000","last_clean":"2024-09-05T13:19:08.430080+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:08.430080+0000","last_undegraded":"2024-09-05T13:19:08.430080+0000","last_fullsized":"2024-09-05T13:19:08.430080+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25288704,"num_objects":6174,"num_object_clones":0,"num_object_copies":12348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12590,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14876","reported_seq":15005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:08.451280+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:08.451280+0000","last_peered":"2024-09-05T13:19:08.451280+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:08.451280+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25882648,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12687,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":216},{"shards":"2","objects":1828}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14724","reported_seq":14795,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:08.405322+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:08.405322+0000","last_peered":"2024-09-05T13:19:08.405322+0000","last_clean":"2024-09-05T13:19:08.405322+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:08.405322+0000","last_undegraded":"2024-09-05T13:19:08.405322+0000","last_fullsized":"2024-09-05T13:19:08.405322+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25812992,"num_objects":6302,"num_object_clones":0,"num_object_copies":12604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14724,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14704","reported_seq":14839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:09.471995+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:09.471995+0000","last_peered":"2024-09-05T13:19:09.471995+0000","last_clean":"2024-09-05T13:19:09.471995+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:09.471995+0000","last_undegraded":"2024-09-05T13:19:09.471995+0000","last_fullsized":"2024-09-05T13:19:09.471995+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25309184,"num_objects":6179,"num_object_clones":0,"num_object_copies":12358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11244,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14850","reported_seq":14989,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.516379+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:09.516379+0000","last_peered":"2024-09-05T13:19:09.516379+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:09.516379+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11305,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1854},{"shards":"0,2","objects":181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14570","reported_seq":21931,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.573917+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:09.573917+0000","last_peered":"2024-09-05T13:19:09.573917+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:09.573917+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10889,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3610,"num_bytes_recovered":14778368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5230},{"shards":"0,2","objects":152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14841","reported_seq":14974,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:09.573531+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:09.573531+0000","last_peered":"2024-09-05T13:19:09.573531+0000","last_clean":"2024-09-05T13:19:09.573531+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:09.573531+0000","last_undegraded":"2024-09-05T13:19:09.573531+0000","last_fullsized":"2024-09-05T13:19:09.573531+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11451,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14800","reported_seq":14936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:08.436987+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:08.436987+0000","last_peered":"2024-09-05T13:19:08.436987+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:08.436987+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25776128,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12454,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":201},{"shards":"2","objects":1842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14727","reported_seq":14865,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.485345+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:09.485345+0000","last_peered":"2024-09-05T13:19:09.485345+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:09.485345+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25325568,"num_objects":6183,"num_object_clones":0,"num_object_copies":12366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11094,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1729},{"shards":"0,2","objects":186}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14617","reported_seq":23519,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:08.467112+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:08.467112+0000","last_peered":"2024-09-05T13:19:08.467112+0000","last_clean":"2024-09-05T13:19:08.467112+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:08.467112+0000","last_undegraded":"2024-09-05T13:19:08.467112+0000","last_fullsized":"2024-09-05T13:19:08.467112+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25780224,"num_objects":6294,"num_object_clones":0,"num_object_copies":12588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12212,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14481","reported_seq":14605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:08.411816+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:08.411816+0000","last_peered":"2024-09-05T13:19:08.411816+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:08.411816+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24989696,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11990,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":200},{"shards":"2","objects":1714}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14673","reported_seq":14813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.542869+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:09.542869+0000","last_peered":"2024-09-05T13:19:09.542869+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:09.542869+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25415680,"num_objects":6205,"num_object_clones":0,"num_object_copies":12410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10824,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1786},{"shards":"0,2","objects":186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14829","reported_seq":15017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.509864+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:09.509864+0000","last_peered":"2024-09-05T13:19:09.509864+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:09.509864+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25702400,"num_objects":6275,"num_object_clones":0,"num_object_copies":12550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10728,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1831},{"shards":"0,2","objects":171}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14912","reported_seq":15052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.550011+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:09.550011+0000","last_peered":"2024-09-05T13:19:09.550011+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:09.550011+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11042,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1602},{"shards":"0,2","objects":196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14969","reported_seq":15110,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.472442+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:09.472442+0000","last_peered":"2024-09-05T13:19:09.472442+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:09.472442+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489891,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11637,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1659},{"shards":"0,2","objects":198}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":407945723,"num_objects":99598,"num_object_clones":0,"num_object_copies":199196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":99598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191589,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8002,"num_bytes_recovered":32759808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1026523136,"data_stored":1026439119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646500,"kb_used_data":344084,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93725340,"statfs":{"total":96636764160,"available":95974748160,"internally_reserved":0,"allocated":352342016,"data_stored":352082459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,3,3,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784780,"kb_used_data":369980,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93587060,"statfs":{"total":96636764160,"available":95833149440,"internally_reserved":0,"allocated":378859520,"data_stored":378599958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,7,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615668,"kb_used_data":291040,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93756172,"statfs":{"total":96636764160,"available":96006320128,"internally_reserved":0,"allocated":298024960,"data_stored":297696292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377761792,"data_stored":377758179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297570304,"data_stored":297497585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351191040,"data_stored":351183355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:12.166 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:12.695 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:12.697 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:12.719 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-05T13:19:11.667213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":407945742,"num_objects":99600,"num_object_clones":0,"num_object_copies":199200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":99600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191591,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8003,"num_bytes_recovered":32759827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046948,"kb_used_data":1005104,"kb_used_omap":12,"kb_used_meta":1041779,"kb_avail":281068572,"statfs":{"total":289910292480,"available":287814217728,"internally_reserved":0,"allocated":1029226496,"data_stored":1028378709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1066781702},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,4,15,5,14,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18395136,"num_objects":-4491,"num_object_clones":0,"num_object_copies":-8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":-4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4491,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006074"},"pg_stats":[{"pgid":"2.f","version":"45'14718","reported_seq":14794,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.554063+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:09.554063+0000","last_peered":"2024-09-05T13:19:09.554063+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:09.554063+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25616384,"num_objects":6254,"num_object_clones":0,"num_object_copies":12508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833},{"shards":"0,2","objects":188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14580","reported_seq":14685,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:08.430080+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:08.430080+0000","last_peered":"2024-09-05T13:19:08.430080+0000","last_clean":"2024-09-05T13:19:08.430080+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:08.430080+0000","last_undegraded":"2024-09-05T13:19:08.430080+0000","last_fullsized":"2024-09-05T13:19:08.430080+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25288704,"num_objects":6174,"num_object_clones":0,"num_object_copies":12348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12590,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14876","reported_seq":15005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:08.451280+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:08.451280+0000","last_peered":"2024-09-05T13:19:08.451280+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:08.451280+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25882648,"num_objects":6320,"num_object_clones":0,"num_object_copies":12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12687,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":216},{"shards":"2","objects":1828}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14724","reported_seq":14795,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:08.405322+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:08.405322+0000","last_peered":"2024-09-05T13:19:08.405322+0000","last_clean":"2024-09-05T13:19:08.405322+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:08.405322+0000","last_undegraded":"2024-09-05T13:19:08.405322+0000","last_fullsized":"2024-09-05T13:19:08.405322+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25812992,"num_objects":6302,"num_object_clones":0,"num_object_copies":12604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14724,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14704","reported_seq":14839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:09.471995+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:09.471995+0000","last_peered":"2024-09-05T13:19:09.471995+0000","last_clean":"2024-09-05T13:19:09.471995+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:09.471995+0000","last_undegraded":"2024-09-05T13:19:09.471995+0000","last_fullsized":"2024-09-05T13:19:09.471995+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25309184,"num_objects":6179,"num_object_clones":0,"num_object_copies":12358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11244,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14850","reported_seq":14989,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.516379+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:09.516379+0000","last_peered":"2024-09-05T13:19:09.516379+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:09.516379+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11305,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1854},{"shards":"0,2","objects":181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14570","reported_seq":21931,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.573917+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:09.573917+0000","last_peered":"2024-09-05T13:19:09.573917+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:09.573917+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10889,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3610,"num_bytes_recovered":14778368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5230},{"shards":"0,2","objects":152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14841","reported_seq":14974,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:09.573531+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:09.573531+0000","last_peered":"2024-09-05T13:19:09.573531+0000","last_clean":"2024-09-05T13:19:09.573531+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:09.573531+0000","last_undegraded":"2024-09-05T13:19:09.573531+0000","last_fullsized":"2024-09-05T13:19:09.573531+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11451,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14800","reported_seq":14936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:08.436987+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:08.436987+0000","last_peered":"2024-09-05T13:19:08.436987+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:08.436987+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25776128,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12454,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":201},{"shards":"2","objects":1842}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14727","reported_seq":14865,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.485345+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:09.485345+0000","last_peered":"2024-09-05T13:19:09.485345+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:09.485345+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25325568,"num_objects":6183,"num_object_clones":0,"num_object_copies":12366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11094,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1729},{"shards":"0,2","objects":186}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14617","reported_seq":23519,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:08.467112+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:08.467112+0000","last_peered":"2024-09-05T13:19:08.467112+0000","last_clean":"2024-09-05T13:19:08.467112+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:08.467112+0000","last_undegraded":"2024-09-05T13:19:08.467112+0000","last_fullsized":"2024-09-05T13:19:08.467112+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25780224,"num_objects":6294,"num_object_clones":0,"num_object_copies":12588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12212,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14481","reported_seq":14605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:08.411816+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:08.411816+0000","last_peered":"2024-09-05T13:19:08.411816+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:08.411816+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24989696,"num_objects":6101,"num_object_clones":0,"num_object_copies":12202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11990,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":200},{"shards":"2","objects":1714}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14673","reported_seq":14813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.542869+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:09.542869+0000","last_peered":"2024-09-05T13:19:09.542869+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:09.542869+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25415680,"num_objects":6205,"num_object_clones":0,"num_object_copies":12410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10824,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1786},{"shards":"0,2","objects":186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14829","reported_seq":15017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.509864+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:09.509864+0000","last_peered":"2024-09-05T13:19:09.509864+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:09.509864+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25702400,"num_objects":6275,"num_object_clones":0,"num_object_copies":12550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10728,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1831},{"shards":"0,2","objects":171}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14912","reported_seq":15052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.550011+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:09.550011+0000","last_peered":"2024-09-05T13:19:09.550011+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:09.550011+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11042,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1602},{"shards":"0,2","objects":196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14969","reported_seq":15110,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.472442+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:09.472442+0000","last_peered":"2024-09-05T13:19:09.472442+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:09.472442+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489891,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11637,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1659},{"shards":"0,2","objects":198}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":407945723,"num_objects":99598,"num_object_clones":0,"num_object_copies":199196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":99598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191589,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8002,"num_bytes_recovered":32759808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1026523136,"data_stored":1026439119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646500,"kb_used_data":344084,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93725340,"statfs":{"total":96636764160,"available":95974748160,"internally_reserved":0,"allocated":352342016,"data_stored":352082459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,3,3,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784780,"kb_used_data":369980,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93587060,"statfs":{"total":96636764160,"available":95833149440,"internally_reserved":0,"allocated":378859520,"data_stored":378599958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,7,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615668,"kb_used_data":291040,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93756172,"statfs":{"total":96636764160,"available":96006320128,"internally_reserved":0,"allocated":298024960,"data_stored":297696292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377761792,"data_stored":377758179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297570304,"data_stored":297497585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":351191040,"data_stored":351183355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:12.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:12.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'14718", 'reported_seq': 14794, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:09.554063+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:09.554063+0000', 'last_peered': '2024-09-05T13:19:09.554063+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:09.554063+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25616384, 'num_objects': 6254, 'num_object_clones': 0, 'num_object_copies': 12508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14718, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1833}, {'shards': '0,2', 'objects': 188}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:12.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:12.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'14876", 'reported_seq': 15005, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:08.451280+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:19:08.451280+0000', 'last_peered': '2024-09-05T13:19:08.451280+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:08.451280+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25882648, 'num_objects': 6320, 'num_object_clones': 0, 'num_object_copies': 12640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6320, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12687, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 216}, {'shards': '2', 'objects': 1828}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'14850", 'reported_seq': 14989, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:09.516379+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:09.516379+0000', 'last_peered': '2024-09-05T13:19:09.516379+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:09.516379+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25825280, 'num_objects': 6305, 'num_object_clones': 0, 'num_object_copies': 12610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11305, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1854}, {'shards': '0,2', 'objects': 181}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'14570", 'reported_seq': 21931, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:09.573917+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:19:09.573917+0000', 'last_peered': '2024-09-05T13:19:09.573917+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:19:09.573917+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24813568, 'num_objects': 6058, 'num_object_clones': 0, 'num_object_copies': 12116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 676, 'num_objects_misplaced': 152, 'num_objects_unfound': 0, 'num_objects_dirty': 6058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10889, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3610, 'num_bytes_recovered': 14778368, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5230}, {'shards': '0,2', 'objects': 152}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'14800", 'reported_seq': 14936, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:08.436987+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:08.436987+0000', 'last_peered': '2024-09-05T13:19:08.436987+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:08.436987+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25776128, 'num_objects': 6293, 'num_object_clones': 0, 'num_object_copies': 12586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12454, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 201}, {'shards': '2', 'objects': 1842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'14727", 'reported_seq': 14865, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:09.485345+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:09.485345+0000', 'last_peered': '2024-09-05T13:19:09.485345+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:09.485345+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25325568, 'num_objects': 6183, 'num_object_clones': 0, 'num_object_copies': 12366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11094, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1729}, {'shards': '0,2', 'objects': 186}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'14481", 'reported_seq': 14605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:08.411816+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:08.411816+0000', 'last_peered': '2024-09-05T13:19:08.411816+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:08.411816+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24989696, 'num_objects': 6101, 'num_object_clones': 0, 'num_object_copies': 12202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11990, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 200}, {'shards': '2', 'objects': 1714}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:12.721 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:12.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'14673", 'reported_seq': 14813, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:09.542869+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:09.542869+0000', 'last_peered': '2024-09-05T13:19:09.542869+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:09.542869+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25415680, 'num_objects': 6205, 'num_object_clones': 0, 'num_object_copies': 12410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10824, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1786}, {'shards': '0,2', 'objects': 186}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:12.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:12.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'14829", 'reported_seq': 15017, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:09.509864+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:09.509864+0000', 'last_peered': '2024-09-05T13:19:09.509864+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:09.509864+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25702400, 'num_objects': 6275, 'num_object_clones': 0, 'num_object_copies': 12550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6275, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10728, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1831}, {'shards': '0,2', 'objects': 171}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:12.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:12.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'14912", 'reported_seq': 15052, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:09.550011+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:09.550011+0000', 'last_peered': '2024-09-05T13:19:09.550011+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:09.550011+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25264128, 'num_objects': 6168, 'num_object_clones': 0, 'num_object_copies': 12336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11042, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1602}, {'shards': '0,2', 'objects': 196}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:12.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:12.722 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'14969", 'reported_seq': 15110, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:09.472442+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:09.472442+0000', 'last_peered': '2024-09-05T13:19:09.472442+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:09.472442+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25489891, 'num_objects': 6224, 'num_object_clones': 0, 'num_object_copies': 12448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11637, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1659}, {'shards': '0,2', 'objects': 198}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:12.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:13.393 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:13.394 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45344/199200 objects degraded (22.763%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":480,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":99600,"data_bytes":407945742,"bytes_used":2096074752,"bytes_avail":287814217728,"bytes_total":289910292480,"degraded_objects":45344,"degraded_total":199200,"degraded_ratio":0.22763052208835341,"misplaced_objects":152,"misplaced_total":199200,"misplaced_ratio":0.00076305220883534123,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":97913,"recovering_keys_per_sec":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":374},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:13.415 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:13.416 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:14.001 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:14.028 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-05T13:19:13.668024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":404529678,"num_objects":98766,"num_object_clones":0,"num_object_copies":197532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":98766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192425,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8003,"num_bytes_recovered":32759827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2050232,"kb_used_data":1000072,"kb_used_omap":12,"kb_used_meta":1050099,"kb_avail":281065288,"statfs":{"total":289910292480,"available":287810854912,"internally_reserved":0,"allocated":1024073728,"data_stored":1023225941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1075301382},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,2,12,12,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21811200,"num_objects":-5325,"num_object_clones":0,"num_object_copies":-10650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":-5325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5325,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006174"},"pg_stats":[{"pgid":"2.f","version":"45'14718","reported_seq":14794,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.554063+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:09.554063+0000","last_peered":"2024-09-05T13:19:09.554063+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:09.554063+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25616384,"num_objects":6254,"num_object_clones":0,"num_object_copies":12508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833},{"shards":"0,2","objects":188}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14732","reported_seq":14838,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.441682+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:13.441682+0000","last_peered":"2024-09-05T13:19:13.441682+0000","last_clean":"2024-09-05T13:19:13.441682+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:13.441682+0000","last_undegraded":"2024-09-05T13:19:13.441682+0000","last_fullsized":"2024-09-05T13:19:13.441682+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24666112,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12742,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15016","reported_seq":15146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.459808+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:13.459808+0000","last_peered":"2024-09-05T13:19:13.459808+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:13.459808+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25309208,"num_objects":6180,"num_object_clones":0,"num_object_copies":12360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12827,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":76},{"shards":"2","objects":1688}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14870","reported_seq":14937,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.365989+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:13.365989+0000","last_peered":"2024-09-05T13:19:13.365989+0000","last_clean":"2024-09-05T13:19:13.365989+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:13.365989+0000","last_undegraded":"2024-09-05T13:19:13.365989+0000","last_fullsized":"2024-09-05T13:19:13.365989+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14870,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14704","reported_seq":14839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:09.471995+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:09.471995+0000","last_peered":"2024-09-05T13:19:09.471995+0000","last_clean":"2024-09-05T13:19:09.471995+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:09.471995+0000","last_undegraded":"2024-09-05T13:19:09.471995+0000","last_fullsized":"2024-09-05T13:19:09.471995+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25309184,"num_objects":6179,"num_object_clones":0,"num_object_copies":12358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11244,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14850","reported_seq":14989,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.516379+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:09.516379+0000","last_peered":"2024-09-05T13:19:09.516379+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:09.516379+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11305,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1854},{"shards":"0,2","objects":181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14570","reported_seq":21931,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.573917+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:09.573917+0000","last_peered":"2024-09-05T13:19:09.573917+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:09.573917+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":676,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10889,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3610,"num_bytes_recovered":14778368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5230},{"shards":"0,2","objects":152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14841","reported_seq":14974,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:09.573531+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:09.573531+0000","last_peered":"2024-09-05T13:19:09.573531+0000","last_clean":"2024-09-05T13:19:09.573531+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:09.573531+0000","last_undegraded":"2024-09-05T13:19:09.573531+0000","last_fullsized":"2024-09-05T13:19:09.573531+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11451,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14936","reported_seq":15072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.387191+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:13.387191+0000","last_peered":"2024-09-05T13:19:13.387191+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:13.387191+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12590,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":65},{"shards":"2","objects":1706}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14727","reported_seq":14865,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.485345+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:09.485345+0000","last_peered":"2024-09-05T13:19:09.485345+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:09.485345+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25325568,"num_objects":6183,"num_object_clones":0,"num_object_copies":12366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11094,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1729},{"shards":"0,2","objects":186}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14759","reported_seq":23661,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.468730+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:13.468730+0000","last_peered":"2024-09-05T13:19:13.468730+0000","last_clean":"2024-09-05T13:19:13.468730+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:13.468730+0000","last_undegraded":"2024-09-05T13:19:13.468730+0000","last_fullsized":"2024-09-05T13:19:13.468730+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12354,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14599","reported_seq":14723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.424800+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:13.424800+0000","last_peered":"2024-09-05T13:19:13.424800+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:13.424800+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":82},{"shards":"2","objects":1596}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14673","reported_seq":14813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.542869+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:09.542869+0000","last_peered":"2024-09-05T13:19:09.542869+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:09.542869+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25415680,"num_objects":6205,"num_object_clones":0,"num_object_copies":12410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10824,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1786},{"shards":"0,2","objects":186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14829","reported_seq":15017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.509864+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:09.509864+0000","last_peered":"2024-09-05T13:19:09.509864+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:09.509864+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25702400,"num_objects":6275,"num_object_clones":0,"num_object_copies":12550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10728,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1831},{"shards":"0,2","objects":171}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14912","reported_seq":15052,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.550011+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:09.550011+0000","last_peered":"2024-09-05T13:19:09.550011+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:09.550011+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25264128,"num_objects":6168,"num_object_clones":0,"num_object_copies":12336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11042,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1602},{"shards":"0,2","objects":196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14969","reported_seq":15110,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:09.472442+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:09.472442+0000","last_peered":"2024-09-05T13:19:09.472442+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:09.472442+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489891,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11637,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1659},{"shards":"0,2","objects":198}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":404529659,"num_objects":98764,"num_object_clones":0,"num_object_copies":197528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45344,"num_objects_misplaced":152,"num_objects_unfound":0,"num_objects_dirty":98764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192423,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8002,"num_bytes_recovered":32759808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021378560,"data_stored":1021294543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646156,"kb_used_data":339580,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93725684,"statfs":{"total":96636764160,"available":95975100416,"internally_reserved":0,"allocated":347729920,"data_stored":347470363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784780,"kb_used_data":369980,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93587060,"statfs":{"total":96636764160,"available":95833149440,"internally_reserved":0,"allocated":378859520,"data_stored":378599958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,7,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619296,"kb_used_data":290512,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752544,"statfs":{"total":96636764160,"available":96002605056,"internally_reserved":0,"allocated":297484288,"data_stored":297155620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,7,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377761792,"data_stored":377758179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296955904,"data_stored":296883185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346660864,"data_stored":346653179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:17.030 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:17.594 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:17.594 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:17.626 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-05T13:19:15.668900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398615054,"num_objects":97322,"num_object_clones":0,"num_object_copies":194644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193869,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8157,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047452,"kb_used_data":993128,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281068068,"statfs":{"total":289910292480,"available":287813701632,"internally_reserved":0,"allocated":1016963072,"data_stored":1016115285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1079561222},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,11,11,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24317952,"num_objects":-5937,"num_object_clones":0,"num_object_copies":-11874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-439,"num_objects_unfound":0,"num_objects_dirty":-5937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5937,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":441,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006615"},"pg_stats":[{"pgid":"2.f","version":"45'14861","reported_seq":14937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.544796+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:14.544796+0000","last_peered":"2024-09-05T13:19:14.544796+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:14.544796+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25030656,"num_objects":6111,"num_object_clones":0,"num_object_copies":12222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1690},{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14732","reported_seq":14838,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.441682+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:13.441682+0000","last_peered":"2024-09-05T13:19:13.441682+0000","last_clean":"2024-09-05T13:19:13.441682+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:13.441682+0000","last_undegraded":"2024-09-05T13:19:13.441682+0000","last_fullsized":"2024-09-05T13:19:13.441682+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24666112,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12742,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15016","reported_seq":15146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.459808+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:13.459808+0000","last_peered":"2024-09-05T13:19:13.459808+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:13.459808+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25309208,"num_objects":6180,"num_object_clones":0,"num_object_copies":12360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12827,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":76},{"shards":"2","objects":1688}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14870","reported_seq":14937,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.365989+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:13.365989+0000","last_peered":"2024-09-05T13:19:13.365989+0000","last_clean":"2024-09-05T13:19:13.365989+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:13.365989+0000","last_undegraded":"2024-09-05T13:19:13.365989+0000","last_fullsized":"2024-09-05T13:19:13.365989+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14870,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14835","reported_seq":14970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:14.503956+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:14.503956+0000","last_peered":"2024-09-05T13:19:14.503956+0000","last_clean":"2024-09-05T13:19:14.503956+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:14.503956+0000","last_undegraded":"2024-09-05T13:19:14.503956+0000","last_fullsized":"2024-09-05T13:19:14.503956+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11375,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14992","reported_seq":15131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.601111+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:14.601111+0000","last_peered":"2024-09-05T13:19:14.601111+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:14.601111+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11447,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1712},{"shards":"0,2","objects":39}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14722","reported_seq":22392,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.621383+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:14.621383+0000","last_peered":"2024-09-05T13:19:14.621383+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:14.621383+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11041,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3764,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14988","reported_seq":15121,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:14.591126+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:14.591126+0000","last_peered":"2024-09-05T13:19:14.591126+0000","last_clean":"2024-09-05T13:19:14.591126+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:14.591126+0000","last_undegraded":"2024-09-05T13:19:14.591126+0000","last_fullsized":"2024-09-05T13:19:14.591126+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051136,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11598,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14936","reported_seq":15072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.387191+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:13.387191+0000","last_peered":"2024-09-05T13:19:13.387191+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:13.387191+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12590,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":65},{"shards":"2","objects":1706}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14869","reported_seq":15008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.549126+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:14.549126+0000","last_peered":"2024-09-05T13:19:14.549126+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:14.549126+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24743936,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11236,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1587},{"shards":"0,2","objects":44}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14759","reported_seq":23661,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.468730+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:13.468730+0000","last_peered":"2024-09-05T13:19:13.468730+0000","last_clean":"2024-09-05T13:19:13.468730+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:13.468730+0000","last_undegraded":"2024-09-05T13:19:13.468730+0000","last_fullsized":"2024-09-05T13:19:13.468730+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12354,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14599","reported_seq":14723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.424800+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:13.424800+0000","last_peered":"2024-09-05T13:19:13.424800+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:13.424800+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":82},{"shards":"2","objects":1596}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14825","reported_seq":14966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.384097+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:14.384097+0000","last_peered":"2024-09-05T13:19:14.384097+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:14.384097+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10976,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1634},{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14955","reported_seq":15143,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.579441+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:14.579441+0000","last_peered":"2024-09-05T13:19:14.579441+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:14.579441+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15070","reported_seq":15210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.506142+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:14.506142+0000","last_peered":"2024-09-05T13:19:14.506142+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:14.506142+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24616960,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1444},{"shards":"0,2","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15120","reported_seq":15254,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.504948+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:14.504948+0000","last_peered":"2024-09-05T13:19:14.504948+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:14.504948+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24871395,"num_objects":6073,"num_object_clones":0,"num_object_copies":12146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11788,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1508},{"shards":"0,2","objects":47}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398615035,"num_objects":97320,"num_object_clones":0,"num_object_copies":194640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193867,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8156,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1014251520,"data_stored":1014167503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646156,"kb_used_data":339580,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93725684,"statfs":{"total":96636764160,"available":95975100416,"internally_reserved":0,"allocated":347729920,"data_stored":347470363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782000,"kb_used_data":363036,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93589840,"statfs":{"total":96636764160,"available":95835996160,"internally_reserved":0,"allocated":371748864,"data_stored":371489302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619296,"kb_used_data":290512,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752544,"statfs":{"total":96636764160,"available":96002605056,"internally_reserved":0,"allocated":297484288,"data_stored":297155620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,7,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370634752,"data_stored":370631139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296955904,"data_stored":296883185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346660864,"data_stored":346653179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:17.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:18.205 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:18.205 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:18.235 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-05T13:19:17.669721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398615054,"num_objects":97322,"num_object_clones":0,"num_object_copies":194644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193869,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8157,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047452,"kb_used_data":993128,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281068068,"statfs":{"total":289910292480,"available":287813701632,"internally_reserved":0,"allocated":1016963072,"data_stored":1016115285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1079561222},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,11,11,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18481152,"num_objects":-4512,"num_object_clones":0,"num_object_copies":-9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":-4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4512,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006741"},"pg_stats":[{"pgid":"2.f","version":"45'14861","reported_seq":14937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.544796+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:14.544796+0000","last_peered":"2024-09-05T13:19:14.544796+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:14.544796+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25030656,"num_objects":6111,"num_object_clones":0,"num_object_copies":12222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1690},{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14732","reported_seq":14838,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.441682+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:13.441682+0000","last_peered":"2024-09-05T13:19:13.441682+0000","last_clean":"2024-09-05T13:19:13.441682+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:13.441682+0000","last_undegraded":"2024-09-05T13:19:13.441682+0000","last_fullsized":"2024-09-05T13:19:13.441682+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24666112,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12742,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15016","reported_seq":15146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.459808+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:13.459808+0000","last_peered":"2024-09-05T13:19:13.459808+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:13.459808+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25309208,"num_objects":6180,"num_object_clones":0,"num_object_copies":12360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12827,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":76},{"shards":"2","objects":1688}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14870","reported_seq":14937,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.365989+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:13.365989+0000","last_peered":"2024-09-05T13:19:13.365989+0000","last_clean":"2024-09-05T13:19:13.365989+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:13.365989+0000","last_undegraded":"2024-09-05T13:19:13.365989+0000","last_fullsized":"2024-09-05T13:19:13.365989+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14870,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14835","reported_seq":14970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:14.503956+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:14.503956+0000","last_peered":"2024-09-05T13:19:14.503956+0000","last_clean":"2024-09-05T13:19:14.503956+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:14.503956+0000","last_undegraded":"2024-09-05T13:19:14.503956+0000","last_fullsized":"2024-09-05T13:19:14.503956+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11375,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14992","reported_seq":15131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.601111+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:14.601111+0000","last_peered":"2024-09-05T13:19:14.601111+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:14.601111+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11447,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1712},{"shards":"0,2","objects":39}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14722","reported_seq":22392,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.621383+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:14.621383+0000","last_peered":"2024-09-05T13:19:14.621383+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:14.621383+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11041,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3764,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14988","reported_seq":15121,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:14.591126+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:14.591126+0000","last_peered":"2024-09-05T13:19:14.591126+0000","last_clean":"2024-09-05T13:19:14.591126+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:14.591126+0000","last_undegraded":"2024-09-05T13:19:14.591126+0000","last_fullsized":"2024-09-05T13:19:14.591126+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051136,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11598,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14936","reported_seq":15072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.387191+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:13.387191+0000","last_peered":"2024-09-05T13:19:13.387191+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:13.387191+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12590,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":65},{"shards":"2","objects":1706}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14869","reported_seq":15008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.549126+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:14.549126+0000","last_peered":"2024-09-05T13:19:14.549126+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:14.549126+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24743936,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11236,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1587},{"shards":"0,2","objects":44}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14759","reported_seq":23661,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.468730+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:13.468730+0000","last_peered":"2024-09-05T13:19:13.468730+0000","last_clean":"2024-09-05T13:19:13.468730+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:13.468730+0000","last_undegraded":"2024-09-05T13:19:13.468730+0000","last_fullsized":"2024-09-05T13:19:13.468730+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12354,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14599","reported_seq":14723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.424800+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:13.424800+0000","last_peered":"2024-09-05T13:19:13.424800+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:13.424800+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":82},{"shards":"2","objects":1596}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14825","reported_seq":14966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.384097+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:14.384097+0000","last_peered":"2024-09-05T13:19:14.384097+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:14.384097+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10976,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1634},{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14955","reported_seq":15143,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.579441+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:14.579441+0000","last_peered":"2024-09-05T13:19:14.579441+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:14.579441+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15070","reported_seq":15210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.506142+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:14.506142+0000","last_peered":"2024-09-05T13:19:14.506142+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:14.506142+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24616960,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1444},{"shards":"0,2","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15120","reported_seq":15254,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.504948+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:14.504948+0000","last_peered":"2024-09-05T13:19:14.504948+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:14.504948+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24871395,"num_objects":6073,"num_object_clones":0,"num_object_copies":12146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11788,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1508},{"shards":"0,2","objects":47}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398615035,"num_objects":97320,"num_object_clones":0,"num_object_copies":194640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193867,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8156,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1014251520,"data_stored":1014167503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646156,"kb_used_data":339580,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93725684,"statfs":{"total":96636764160,"available":95975100416,"internally_reserved":0,"allocated":347729920,"data_stored":347470363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782000,"kb_used_data":363036,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93589840,"statfs":{"total":96636764160,"available":95835996160,"internally_reserved":0,"allocated":371748864,"data_stored":371489302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619296,"kb_used_data":290512,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752544,"statfs":{"total":96636764160,"available":96002605056,"internally_reserved":0,"allocated":297484288,"data_stored":297155620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,7,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370634752,"data_stored":370631139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296955904,"data_stored":296883185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346660864,"data_stored":346653179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:18.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:18.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'14861", 'reported_seq': 14937, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:14.544796+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:14.544796+0000', 'last_peered': '2024-09-05T13:19:14.544796+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:14.544796+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25030656, 'num_objects': 6111, 'num_object_clones': 0, 'num_object_copies': 12222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14861, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1690}, {'shards': '0,2', 'objects': 45}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:18.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:18.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'15016", 'reported_seq': 15146, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:13.459808+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:19:13.459808+0000', 'last_peered': '2024-09-05T13:19:13.459808+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:13.459808+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25309208, 'num_objects': 6180, 'num_object_clones': 0, 'num_object_copies': 12360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6180, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12827, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 76}, {'shards': '2', 'objects': 1688}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:18.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:18.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'14992", 'reported_seq': 15131, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:14.601111+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:14.601111+0000', 'last_peered': '2024-09-05T13:19:14.601111+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:14.601111+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25243648, 'num_objects': 6163, 'num_object_clones': 0, 'num_object_copies': 12326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6163, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11447, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1712}, {'shards': '0,2', 'objects': 39}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:18.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:19:18.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'14722", 'reported_seq': 22392, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:14.621383+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:19:14.621383+0000', 'last_peered': '2024-09-05T13:19:14.621383+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:19:14.621383+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24190976, 'num_objects': 5906, 'num_object_clones': 0, 'num_object_copies': 11812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 674, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11041, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3764, 'num_bytes_recovered': 15405056, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5232}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:18.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:18.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'14936", 'reported_seq': 15072, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:13.387191+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:13.387191+0000', 'last_peered': '2024-09-05T13:19:13.387191+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:13.387191+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25219072, 'num_objects': 6157, 'num_object_clones': 0, 'num_object_copies': 12314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12590, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 65}, {'shards': '2', 'objects': 1706}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:18.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:18.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'14869", 'reported_seq': 15008, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:14.549126+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:14.549126+0000', 'last_peered': '2024-09-05T13:19:14.549126+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:14.549126+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24743936, 'num_objects': 6041, 'num_object_clones': 0, 'num_object_copies': 12082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11236, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1587}, {'shards': '0,2', 'objects': 44}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:18.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:18.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'14599", 'reported_seq': 14723, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:13.424800+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:13.424800+0000', 'last_peered': '2024-09-05T13:19:13.424800+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:13.424800+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24506368, 'num_objects': 5983, 'num_object_clones': 0, 'num_object_copies': 11966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12108, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 82}, {'shards': '2', 'objects': 1596}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:18.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:18.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'14825", 'reported_seq': 14966, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:14.384097+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:14.384097+0000', 'last_peered': '2024-09-05T13:19:14.384097+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:14.384097+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24793088, 'num_objects': 6053, 'num_object_clones': 0, 'num_object_copies': 12106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6053, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10976, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1634}, {'shards': '0,2', 'objects': 34}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:18.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:18.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'14955", 'reported_seq': 15143, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:14.579441+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:14.579441+0000', 'last_peered': '2024-09-05T13:19:14.579441+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:14.579441+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25186304, 'num_objects': 6149, 'num_object_clones': 0, 'num_object_copies': 12298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10854, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1705}, {'shards': '0,2', 'objects': 45}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:18.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:18.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'15070", 'reported_seq': 15210, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:14.506142+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:14.506142+0000', 'last_peered': '2024-09-05T13:19:14.506142+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:14.506142+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24616960, 'num_objects': 6010, 'num_object_clones': 0, 'num_object_copies': 12020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6010, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11200, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1444}, {'shards': '0,2', 'objects': 38}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:18.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:18.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'15120", 'reported_seq': 15254, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:14.504948+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:14.504948+0000', 'last_peered': '2024-09-05T13:19:14.504948+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:14.504948+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24871395, 'num_objects': 6073, 'num_object_clones': 0, 'num_object_copies': 12146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6073, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11788, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1508}, {'shards': '0,2', 'objects': 47}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:18.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:18.918 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:18.918 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45342/194644 objects degraded (23.295%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":485,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":97322,"data_bytes":398615054,"bytes_used":2096590848,"bytes_avail":287813701632,"bytes_total":289910292480,"degraded_objects":45342,"degraded_total":194644,"degraded_ratio":0.23294835700047264,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":104389,"recovering_keys_per_sec":0,"num_objects_recovered":307,"num_bytes_recovered":1253376,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":375},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:18.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:18.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:19.507 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:19.509 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:19.529 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-05T13:19:17.669721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398615054,"num_objects":97322,"num_object_clones":0,"num_object_copies":194644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193869,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8157,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047452,"kb_used_data":993128,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281068068,"statfs":{"total":289910292480,"available":287813701632,"internally_reserved":0,"allocated":1016963072,"data_stored":1016115285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1079561222},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,11,11,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18481152,"num_objects":-4512,"num_object_clones":0,"num_object_copies":-9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":-4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4512,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006741"},"pg_stats":[{"pgid":"2.f","version":"45'14861","reported_seq":14937,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.544796+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:14.544796+0000","last_peered":"2024-09-05T13:19:14.544796+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:14.544796+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25030656,"num_objects":6111,"num_object_clones":0,"num_object_copies":12222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14861,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1690},{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14732","reported_seq":14838,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.441682+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:13.441682+0000","last_peered":"2024-09-05T13:19:13.441682+0000","last_clean":"2024-09-05T13:19:13.441682+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:13.441682+0000","last_undegraded":"2024-09-05T13:19:13.441682+0000","last_fullsized":"2024-09-05T13:19:13.441682+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24666112,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12742,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15016","reported_seq":15146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.459808+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:13.459808+0000","last_peered":"2024-09-05T13:19:13.459808+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:13.459808+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25309208,"num_objects":6180,"num_object_clones":0,"num_object_copies":12360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12827,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":76},{"shards":"2","objects":1688}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14870","reported_seq":14937,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.365989+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:13.365989+0000","last_peered":"2024-09-05T13:19:13.365989+0000","last_clean":"2024-09-05T13:19:13.365989+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:13.365989+0000","last_undegraded":"2024-09-05T13:19:13.365989+0000","last_fullsized":"2024-09-05T13:19:13.365989+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14870,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14835","reported_seq":14970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:14.503956+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:14.503956+0000","last_peered":"2024-09-05T13:19:14.503956+0000","last_clean":"2024-09-05T13:19:14.503956+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:14.503956+0000","last_undegraded":"2024-09-05T13:19:14.503956+0000","last_fullsized":"2024-09-05T13:19:14.503956+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11375,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14992","reported_seq":15131,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.601111+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:14.601111+0000","last_peered":"2024-09-05T13:19:14.601111+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:14.601111+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25243648,"num_objects":6163,"num_object_clones":0,"num_object_copies":12326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11447,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1712},{"shards":"0,2","objects":39}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14722","reported_seq":22392,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.621383+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:14.621383+0000","last_peered":"2024-09-05T13:19:14.621383+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:14.621383+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11041,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3764,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5232}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14988","reported_seq":15121,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:14.591126+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:14.591126+0000","last_peered":"2024-09-05T13:19:14.591126+0000","last_clean":"2024-09-05T13:19:14.591126+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:14.591126+0000","last_undegraded":"2024-09-05T13:19:14.591126+0000","last_fullsized":"2024-09-05T13:19:14.591126+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051136,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11598,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200621+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:16:09.200621+0000","last_peered":"2024-09-05T13:16:09.200621+0000","last_clean":"2024-09-05T13:16:09.200621+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:16:09.200621+0000","last_undegraded":"2024-09-05T13:16:09.200621+0000","last_fullsized":"2024-09-05T13:16:09.200621+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14936","reported_seq":15072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.387191+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:13.387191+0000","last_peered":"2024-09-05T13:19:13.387191+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:13.387191+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25219072,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12590,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":65},{"shards":"2","objects":1706}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14869","reported_seq":15008,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.549126+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:14.549126+0000","last_peered":"2024-09-05T13:19:14.549126+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:14.549126+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24743936,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11236,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1587},{"shards":"0,2","objects":44}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14759","reported_seq":23661,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:13.468730+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:13.468730+0000","last_peered":"2024-09-05T13:19:13.468730+0000","last_clean":"2024-09-05T13:19:13.468730+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:13.468730+0000","last_undegraded":"2024-09-05T13:19:13.468730+0000","last_fullsized":"2024-09-05T13:19:13.468730+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12354,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14599","reported_seq":14723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:13.424800+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:13.424800+0000","last_peered":"2024-09-05T13:19:13.424800+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:13.424800+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":82},{"shards":"2","objects":1596}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14825","reported_seq":14966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.384097+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:14.384097+0000","last_peered":"2024-09-05T13:19:14.384097+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:14.384097+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24793088,"num_objects":6053,"num_object_clones":0,"num_object_copies":12106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10976,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1634},{"shards":"0,2","objects":34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14955","reported_seq":15143,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.579441+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:14.579441+0000","last_peered":"2024-09-05T13:19:14.579441+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:14.579441+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25186304,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":45}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15070","reported_seq":15210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.506142+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:14.506142+0000","last_peered":"2024-09-05T13:19:14.506142+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:14.506142+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24616960,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1444},{"shards":"0,2","objects":38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15120","reported_seq":15254,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:14.504948+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:14.504948+0000","last_peered":"2024-09-05T13:19:14.504948+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:14.504948+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24871395,"num_objects":6073,"num_object_clones":0,"num_object_copies":12146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11788,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1508},{"shards":"0,2","objects":47}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398615035,"num_objects":97320,"num_object_clones":0,"num_object_copies":194640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193867,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8156,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1014251520,"data_stored":1014167503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646156,"kb_used_data":339580,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93725684,"statfs":{"total":96636764160,"available":95975100416,"internally_reserved":0,"allocated":347729920,"data_stored":347470363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782000,"kb_used_data":363036,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93589840,"statfs":{"total":96636764160,"available":95835996160,"internally_reserved":0,"allocated":371748864,"data_stored":371489302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619296,"kb_used_data":290512,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752544,"statfs":{"total":96636764160,"available":96002605056,"internally_reserved":0,"allocated":297484288,"data_stored":297155620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,7,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370634752,"data_stored":370631139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296955904,"data_stored":296883185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346660864,"data_stored":346653179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:22.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:23.097 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:23.099 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:23.117 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-05T13:19:21.671302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":389100046,"num_objects":94999,"num_object_clones":0,"num_object_copies":189998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196192,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8311,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2038332,"kb_used_data":978420,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281077188,"statfs":{"total":289910292480,"available":287823040512,"internally_reserved":0,"allocated":1001902080,"data_stored":1001054293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1085262854},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,5,8,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18845696,"num_objects":-4601,"num_object_clones":0,"num_object_copies":-9202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-156,"num_objects_misplaced":-152,"num_objects_unfound":0,"num_objects_dirty":-4601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4601,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":626688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004884"},"pg_stats":[{"pgid":"2.f","version":"45'15000","reported_seq":15076,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.488052+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:19.488052+0000","last_peered":"2024-09-05T13:19:19.488052+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:19.488052+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24461312,"num_objects":5972,"num_object_clones":0,"num_object_copies":11944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15000,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1551}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14908","reported_seq":15014,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:18.424864+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:18.424864+0000","last_peered":"2024-09-05T13:19:18.424864+0000","last_clean":"2024-09-05T13:19:18.424864+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:18.424864+0000","last_undegraded":"2024-09-05T13:19:18.424864+0000","last_fullsized":"2024-09-05T13:19:18.424864+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23945216,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15174","reported_seq":15304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:18.430935+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:18.430935+0000","last_peered":"2024-09-05T13:19:18.430935+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:18.430935+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24662040,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12985,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15031","reported_seq":15102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:18.475639+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:18.475639+0000","last_peered":"2024-09-05T13:19:18.475639+0000","last_clean":"2024-09-05T13:19:18.475639+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:18.475639+0000","last_undegraded":"2024-09-05T13:19:18.475639+0000","last_fullsized":"2024-09-05T13:19:18.475639+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24555520,"num_objects":5995,"num_object_clones":0,"num_object_copies":11990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15031,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14967","reported_seq":15102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:19.635698+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:19.635698+0000","last_peered":"2024-09-05T13:19:19.635698+0000","last_clean":"2024-09-05T13:19:19.635698+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:19.635698+0000","last_undegraded":"2024-09-05T13:19:19.635698+0000","last_fullsized":"2024-09-05T13:19:19.635698+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24231936,"num_objects":5916,"num_object_clones":0,"num_object_copies":11832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11507,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15137","reported_seq":15276,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.596518+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:19.596518+0000","last_peered":"2024-09-05T13:19:19.596518+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:19.596518+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24649728,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11592,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14861","reported_seq":22839,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.619897+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:19.619897+0000","last_peered":"2024-09-05T13:19:19.619897+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:19.619897+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23621632,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11180,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3918,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15128","reported_seq":15260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:19.554219+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:19.554219+0000","last_peered":"2024-09-05T13:19:19.554219+0000","last_clean":"2024-09-05T13:19:19.554219+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:19.554219+0000","last_undegraded":"2024-09-05T13:19:19.554219+0000","last_fullsized":"2024-09-05T13:19:19.554219+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24477696,"num_objects":5976,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11738,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15084","reported_seq":15220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:18.469806+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:18.469806+0000","last_peered":"2024-09-05T13:19:18.469806+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:18.469806+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24612864,"num_objects":6009,"num_object_clones":0,"num_object_copies":12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12738,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1558}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15012","reported_seq":15151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.595807+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:19.595807+0000","last_peered":"2024-09-05T13:19:19.595807+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:19.595807+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24158208,"num_objects":5898,"num_object_clones":0,"num_object_copies":11796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11379,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1444}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14907","reported_seq":23809,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:18.333516+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:18.333516+0000","last_peered":"2024-09-05T13:19:18.333516+0000","last_clean":"2024-09-05T13:19:18.333516+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:18.333516+0000","last_undegraded":"2024-09-05T13:19:18.333516+0000","last_fullsized":"2024-09-05T13:19:18.333516+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12502,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14744","reported_seq":14868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:18.307243+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:18.307243+0000","last_peered":"2024-09-05T13:19:18.307243+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:18.307243+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23912448,"num_objects":5838,"num_object_clones":0,"num_object_copies":11676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12253,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1451}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14961","reported_seq":15100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.618305+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:19.618305+0000","last_peered":"2024-09-05T13:19:19.618305+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:19.618305+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24236032,"num_objects":5917,"num_object_clones":0,"num_object_copies":11834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11112,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1498}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15098","reported_seq":15286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.532559+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:19.532559+0000","last_peered":"2024-09-05T13:19:19.532559+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:19.532559+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600576,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10997,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15201","reported_seq":15340,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.557622+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:19.557622+0000","last_peered":"2024-09-05T13:19:19.557622+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:19.557622+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24080384,"num_objects":5879,"num_object_clones":0,"num_object_copies":11758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11331,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15259","reported_seq":15400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.619550+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:19.619550+0000","last_peered":"2024-09-05T13:19:19.619550+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:19.619550+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24302051,"num_objects":5934,"num_object_clones":0,"num_object_copies":11868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":389100027,"num_objects":94997,"num_object_clones":0,"num_object_copies":189994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196190,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8310,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":999055360,"data_stored":998971343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645552,"kb_used_data":334816,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93726288,"statfs":{"total":96636764160,"available":95975718912,"internally_reserved":0,"allocated":342851584,"data_stored":342592027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":766856,"kb_used_data":356208,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93604984,"statfs":{"total":96636764160,"available":95851503616,"internally_reserved":0,"allocated":364756992,"data_stored":364497430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625924,"kb_used_data":287396,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93745916,"statfs":{"total":96636764160,"available":95995817984,"internally_reserved":0,"allocated":294293504,"data_stored":293964836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":363646976,"data_stored":363643363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":293699584,"data_stored":293626865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":341708800,"data_stored":341701115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:23.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:23.681 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:23.706 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-05T13:19:23.672004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385413646,"num_objects":94099,"num_object_clones":0,"num_object_copies":188198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197092,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8311,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2022344,"kb_used_data":966588,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281093176,"statfs":{"total":289910292480,"available":287839412224,"internally_reserved":0,"allocated":989786112,"data_stored":988938325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1081003014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,2,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22532096,"num_objects":-5501,"num_object_clones":0,"num_object_copies":-11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-156,"num_objects_misplaced":-152,"num_objects_unfound":0,"num_objects_dirty":-5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5501,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":626688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004790"},"pg_stats":[{"pgid":"2.f","version":"45'15000","reported_seq":15076,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.488052+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:19.488052+0000","last_peered":"2024-09-05T13:19:19.488052+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:19.488052+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24461312,"num_objects":5972,"num_object_clones":0,"num_object_copies":11944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15000,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1551}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15037","reported_seq":15143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.357965+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:23.357965+0000","last_peered":"2024-09-05T13:19:23.357965+0000","last_clean":"2024-09-05T13:19:23.357965+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:23.357965+0000","last_undegraded":"2024-09-05T13:19:23.357965+0000","last_fullsized":"2024-09-05T13:19:23.357965+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13047,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15309","reported_seq":15439,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.466291+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:23.466291+0000","last_peered":"2024-09-05T13:19:23.466291+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:23.466291+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109080,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13120,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1395}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15192","reported_seq":15263,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.467858+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:23.467858+0000","last_peered":"2024-09-05T13:19:23.467858+0000","last_clean":"2024-09-05T13:19:23.467858+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:23.467858+0000","last_undegraded":"2024-09-05T13:19:23.467858+0000","last_fullsized":"2024-09-05T13:19:23.467858+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15192,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14967","reported_seq":15102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:19.635698+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:19.635698+0000","last_peered":"2024-09-05T13:19:19.635698+0000","last_clean":"2024-09-05T13:19:19.635698+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:19.635698+0000","last_undegraded":"2024-09-05T13:19:19.635698+0000","last_fullsized":"2024-09-05T13:19:19.635698+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24231936,"num_objects":5916,"num_object_clones":0,"num_object_copies":11832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11507,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15137","reported_seq":15276,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.596518+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:19.596518+0000","last_peered":"2024-09-05T13:19:19.596518+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:19.596518+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24649728,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11592,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14861","reported_seq":22839,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.619897+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:19.619897+0000","last_peered":"2024-09-05T13:19:19.619897+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:19.619897+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23621632,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11180,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3918,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15128","reported_seq":15260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:19.554219+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:19.554219+0000","last_peered":"2024-09-05T13:19:19.554219+0000","last_clean":"2024-09-05T13:19:19.554219+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:19.554219+0000","last_undegraded":"2024-09-05T13:19:19.554219+0000","last_fullsized":"2024-09-05T13:19:19.554219+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24477696,"num_objects":5976,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11738,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15228","reported_seq":15364,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.343604+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:23.343604+0000","last_peered":"2024-09-05T13:19:23.343604+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:23.343604+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12882,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15012","reported_seq":15151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.595807+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:19.595807+0000","last_peered":"2024-09-05T13:19:19.595807+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:19.595807+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24158208,"num_objects":5898,"num_object_clones":0,"num_object_copies":11796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11379,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1444}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15090","reported_seq":23992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.424616+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:23.424616+0000","last_peered":"2024-09-05T13:19:23.424616+0000","last_clean":"2024-09-05T13:19:23.424616+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:23.424616+0000","last_undegraded":"2024-09-05T13:19:23.424616+0000","last_fullsized":"2024-09-05T13:19:23.424616+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12685,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14892","reported_seq":15015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.442292+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:23.442292+0000","last_peered":"2024-09-05T13:19:23.442292+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:23.442292+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12401,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14961","reported_seq":15100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.618305+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:19.618305+0000","last_peered":"2024-09-05T13:19:19.618305+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:19.618305+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24236032,"num_objects":5917,"num_object_clones":0,"num_object_copies":11834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11112,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1498}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15098","reported_seq":15286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.532559+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:19.532559+0000","last_peered":"2024-09-05T13:19:19.532559+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:19.532559+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600576,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10997,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15201","reported_seq":15340,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.557622+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:19.557622+0000","last_peered":"2024-09-05T13:19:19.557622+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:19.557622+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24080384,"num_objects":5879,"num_object_clones":0,"num_object_copies":11758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11331,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15259","reported_seq":15400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.619550+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:19.619550+0000","last_peered":"2024-09-05T13:19:19.619550+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:19.619550+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24302051,"num_objects":5934,"num_object_clones":0,"num_object_copies":11868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385413627,"num_objects":94097,"num_object_clones":0,"num_object_copies":188194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94097,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197090,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8310,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":986931200,"data_stored":986847183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628284,"kb_used_data":330020,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93743556,"statfs":{"total":96636764160,"available":95993401344,"internally_reserved":0,"allocated":337940480,"data_stored":337680923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":766856,"kb_used_data":356208,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93604984,"statfs":{"total":96636764160,"available":95851503616,"internally_reserved":0,"allocated":364756992,"data_stored":364497430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627204,"kb_used_data":280360,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93744636,"statfs":{"total":96636764160,"available":95994507264,"internally_reserved":0,"allocated":287088640,"data_stored":286759972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":363646976,"data_stored":363643363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286478336,"data_stored":286405617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336805888,"data_stored":336798203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:23.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:23.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'15000", 'reported_seq': 15076, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:19.488052+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:19.488052+0000', 'last_peered': '2024-09-05T13:19:19.488052+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:19.488052+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24461312, 'num_objects': 5972, 'num_object_clones': 0, 'num_object_copies': 11944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15000, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1551}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:23.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:23.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'15309", 'reported_seq': 15439, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:23.466291+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:19:23.466291+0000', 'last_peered': '2024-09-05T13:19:23.466291+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:23.466291+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24109080, 'num_objects': 5887, 'num_object_clones': 0, 'num_object_copies': 11774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5887, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13120, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1395}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:23.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:23.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'15137", 'reported_seq': 15276, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:19.596518+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:19.596518+0000', 'last_peered': '2024-09-05T13:19:19.596518+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:19.596518+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24649728, 'num_objects': 6018, 'num_object_clones': 0, 'num_object_copies': 12036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11592, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:23.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:19:23.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'14861", 'reported_seq': 22839, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:19.619897+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:19:19.619897+0000', 'last_peered': '2024-09-05T13:19:19.619897+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:19:19.619897+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23621632, 'num_objects': 5767, 'num_object_clones': 0, 'num_object_copies': 11534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 520, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11180, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3918, 'num_bytes_recovered': 15405056, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5247}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'15228", 'reported_seq': 15364, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:23.343604+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:23.343604+0000', 'last_peered': '2024-09-05T13:19:23.343604+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:23.343604+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24023040, 'num_objects': 5865, 'num_object_clones': 0, 'num_object_copies': 11730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12882, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1414}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'15012", 'reported_seq': 15151, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:19.595807+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:19.595807+0000', 'last_peered': '2024-09-05T13:19:19.595807+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:19.595807+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24158208, 'num_objects': 5898, 'num_object_clones': 0, 'num_object_copies': 11796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5898, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11379, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1444}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'14892", 'reported_seq': 15015, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:23.442292+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:23.442292+0000', 'last_peered': '2024-09-05T13:19:23.442292+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:23.442292+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23306240, 'num_objects': 5690, 'num_object_clones': 0, 'num_object_copies': 11380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12401, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'14961", 'reported_seq': 15100, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:19.618305+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:19.618305+0000', 'last_peered': '2024-09-05T13:19:19.618305+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:19.618305+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24236032, 'num_objects': 5917, 'num_object_clones': 0, 'num_object_copies': 11834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5917, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11112, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1498}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'15098", 'reported_seq': 15286, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:19.532559+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:19.532559+0000', 'last_peered': '2024-09-05T13:19:19.532559+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:19.532559+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24600576, 'num_objects': 6006, 'num_object_clones': 0, 'num_object_copies': 12012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10997, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1562}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:23.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:23.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'15201", 'reported_seq': 15340, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:19.557622+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:19.557622+0000', 'last_peered': '2024-09-05T13:19:19.557622+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:19.557622+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24080384, 'num_objects': 5879, 'num_object_clones': 0, 'num_object_copies': 11758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11331, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:23.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:23.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'15259", 'reported_seq': 15400, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:19.619550+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:19.619550+0000', 'last_peered': '2024-09-05T13:19:19.619550+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:19.619550+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24302051, 'num_objects': 5934, 'num_object_clones': 0, 'num_object_copies': 11868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5934, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11927, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1369}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:23.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:24.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:24.425 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45188/188198 objects degraded (24.011%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":491,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94099,"data_bytes":385413646,"bytes_used":2070880256,"bytes_avail":287839412224,"bytes_total":289910292480,"degraded_objects":45188,"degraded_total":188198,"degraded_ratio":0.24010882156027166,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":52203,"recovering_keys_per_sec":0,"num_objects_recovered":308,"num_bytes_recovered":626688,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":458},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:24.453 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:24.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:25.054 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-05T13:19:23.672004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385413646,"num_objects":94099,"num_object_clones":0,"num_object_copies":188198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197092,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8311,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2022344,"kb_used_data":966588,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281093176,"statfs":{"total":289910292480,"available":287839412224,"internally_reserved":0,"allocated":989786112,"data_stored":988938325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1081003014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,2,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22532096,"num_objects":-5501,"num_object_clones":0,"num_object_copies":-11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-156,"num_objects_misplaced":-152,"num_objects_unfound":0,"num_objects_dirty":-5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5501,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":626688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004790"},"pg_stats":[{"pgid":"2.f","version":"45'15000","reported_seq":15076,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.488052+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:19.488052+0000","last_peered":"2024-09-05T13:19:19.488052+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:19.488052+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24461312,"num_objects":5972,"num_object_clones":0,"num_object_copies":11944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15000,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1551}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15037","reported_seq":15143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.357965+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:23.357965+0000","last_peered":"2024-09-05T13:19:23.357965+0000","last_clean":"2024-09-05T13:19:23.357965+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:23.357965+0000","last_undegraded":"2024-09-05T13:19:23.357965+0000","last_fullsized":"2024-09-05T13:19:23.357965+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13047,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15309","reported_seq":15439,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.466291+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:23.466291+0000","last_peered":"2024-09-05T13:19:23.466291+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:23.466291+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109080,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13120,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1395}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15192","reported_seq":15263,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.467858+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:23.467858+0000","last_peered":"2024-09-05T13:19:23.467858+0000","last_clean":"2024-09-05T13:19:23.467858+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:23.467858+0000","last_undegraded":"2024-09-05T13:19:23.467858+0000","last_fullsized":"2024-09-05T13:19:23.467858+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15192,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14967","reported_seq":15102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:19.635698+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:19.635698+0000","last_peered":"2024-09-05T13:19:19.635698+0000","last_clean":"2024-09-05T13:19:19.635698+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:19.635698+0000","last_undegraded":"2024-09-05T13:19:19.635698+0000","last_fullsized":"2024-09-05T13:19:19.635698+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24231936,"num_objects":5916,"num_object_clones":0,"num_object_copies":11832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11507,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15137","reported_seq":15276,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.596518+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:19.596518+0000","last_peered":"2024-09-05T13:19:19.596518+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:19.596518+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24649728,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11592,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14861","reported_seq":22839,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.619897+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:19.619897+0000","last_peered":"2024-09-05T13:19:19.619897+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:19.619897+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23621632,"num_objects":5767,"num_object_clones":0,"num_object_copies":11534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":520,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11180,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3918,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15128","reported_seq":15260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:19.554219+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:19.554219+0000","last_peered":"2024-09-05T13:19:19.554219+0000","last_clean":"2024-09-05T13:19:19.554219+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:19.554219+0000","last_undegraded":"2024-09-05T13:19:19.554219+0000","last_fullsized":"2024-09-05T13:19:19.554219+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24477696,"num_objects":5976,"num_object_clones":0,"num_object_copies":11952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11738,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15228","reported_seq":15364,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.343604+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:23.343604+0000","last_peered":"2024-09-05T13:19:23.343604+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:23.343604+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12882,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15012","reported_seq":15151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.595807+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:19.595807+0000","last_peered":"2024-09-05T13:19:19.595807+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:19.595807+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24158208,"num_objects":5898,"num_object_clones":0,"num_object_copies":11796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11379,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1444}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15090","reported_seq":23992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.424616+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:23.424616+0000","last_peered":"2024-09-05T13:19:23.424616+0000","last_clean":"2024-09-05T13:19:23.424616+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:23.424616+0000","last_undegraded":"2024-09-05T13:19:23.424616+0000","last_fullsized":"2024-09-05T13:19:23.424616+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12685,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14892","reported_seq":15015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.442292+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:23.442292+0000","last_peered":"2024-09-05T13:19:23.442292+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:23.442292+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12401,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14961","reported_seq":15100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.618305+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:19.618305+0000","last_peered":"2024-09-05T13:19:19.618305+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:19.618305+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24236032,"num_objects":5917,"num_object_clones":0,"num_object_copies":11834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11112,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1498}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15098","reported_seq":15286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.532559+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:19.532559+0000","last_peered":"2024-09-05T13:19:19.532559+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:19.532559+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600576,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10997,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15201","reported_seq":15340,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.557622+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:19.557622+0000","last_peered":"2024-09-05T13:19:19.557622+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:19.557622+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24080384,"num_objects":5879,"num_object_clones":0,"num_object_copies":11758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11331,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15259","reported_seq":15400,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:19.619550+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:19.619550+0000","last_peered":"2024-09-05T13:19:19.619550+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:19.619550+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24302051,"num_objects":5934,"num_object_clones":0,"num_object_copies":11868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11927,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385413627,"num_objects":94097,"num_object_clones":0,"num_object_copies":188194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94097,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197090,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8310,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":986931200,"data_stored":986847183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628284,"kb_used_data":330020,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93743556,"statfs":{"total":96636764160,"available":95993401344,"internally_reserved":0,"allocated":337940480,"data_stored":337680923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":766856,"kb_used_data":356208,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93604984,"statfs":{"total":96636764160,"available":95851503616,"internally_reserved":0,"allocated":364756992,"data_stored":364497430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627204,"kb_used_data":280360,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93744636,"statfs":{"total":96636764160,"available":95994507264,"internally_reserved":0,"allocated":287088640,"data_stored":286759972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":363646976,"data_stored":363643363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286478336,"data_stored":286405617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336805888,"data_stored":336798203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:28.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:28.618 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:28.620 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:28.646 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-05T13:19:27.673602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":379605518,"num_objects":92681,"num_object_clones":0,"num_object_copies":185362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198510,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8486,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2019628,"kb_used_data":959720,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281095892,"statfs":{"total":289910292480,"available":287842193408,"internally_reserved":0,"allocated":982753280,"data_stored":981905493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1085262854},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,5,8,11,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19009536,"num_objects":-4641,"num_object_clones":0,"num_object_copies":-9282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4641,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004702"},"pg_stats":[{"pgid":"2.f","version":"45'15135","reported_seq":15210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.530977+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:24.530977+0000","last_peered":"2024-09-05T13:19:24.530977+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:24.530977+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15135,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1416}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15037","reported_seq":15143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.357965+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:23.357965+0000","last_peered":"2024-09-05T13:19:23.357965+0000","last_clean":"2024-09-05T13:19:23.357965+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:23.357965+0000","last_undegraded":"2024-09-05T13:19:23.357965+0000","last_fullsized":"2024-09-05T13:19:23.357965+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13047,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15309","reported_seq":15439,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.466291+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:23.466291+0000","last_peered":"2024-09-05T13:19:23.466291+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:23.466291+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109080,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13120,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1395}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15192","reported_seq":15263,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.467858+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:23.467858+0000","last_peered":"2024-09-05T13:19:23.467858+0000","last_clean":"2024-09-05T13:19:23.467858+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:23.467858+0000","last_undegraded":"2024-09-05T13:19:23.467858+0000","last_fullsized":"2024-09-05T13:19:23.467858+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15192,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15106","reported_seq":15241,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:24.637056+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:24.637056+0000","last_peered":"2024-09-05T13:19:24.637056+0000","last_clean":"2024-09-05T13:19:24.637056+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:24.637056+0000","last_undegraded":"2024-09-05T13:19:24.637056+0000","last_fullsized":"2024-09-05T13:19:24.637056+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23662592,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11646,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15290","reported_seq":15429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.598485+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:24.598485+0000","last_peered":"2024-09-05T13:19:24.598485+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:24.598485+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11745,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14970","reported_seq":23298,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.651933+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:24.651933+0000","last_peered":"2024-09-05T13:19:24.651933+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:24.651933+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175168,"num_objects":5658,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11289,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4093,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15293","reported_seq":15425,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:24.652636+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:24.652636+0000","last_peered":"2024-09-05T13:19:24.652636+0000","last_clean":"2024-09-05T13:19:24.652636+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:24.652636+0000","last_undegraded":"2024-09-05T13:19:24.652636+0000","last_fullsized":"2024-09-05T13:19:24.652636+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11903,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15228","reported_seq":15364,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.343604+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:23.343604+0000","last_peered":"2024-09-05T13:19:23.343604+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:23.343604+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12882,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15147","reported_seq":15286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.500374+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:24.500374+0000","last_peered":"2024-09-05T13:19:24.500374+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:24.500374+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11514,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15090","reported_seq":23992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.424616+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:23.424616+0000","last_peered":"2024-09-05T13:19:23.424616+0000","last_clean":"2024-09-05T13:19:23.424616+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:23.424616+0000","last_undegraded":"2024-09-05T13:19:23.424616+0000","last_fullsized":"2024-09-05T13:19:23.424616+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12685,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14892","reported_seq":15015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.442292+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:23.442292+0000","last_peered":"2024-09-05T13:19:23.442292+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:23.442292+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12401,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15108","reported_seq":15249,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.636981+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:24.636981+0000","last_peered":"2024-09-05T13:19:24.636981+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:24.636981+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11259,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15250","reported_seq":15438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.564133+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:24.564133+0000","last_peered":"2024-09-05T13:19:24.564133+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:24.564133+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11149,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15335","reported_seq":15475,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.584250+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:24.584250+0000","last_peered":"2024-09-05T13:19:24.584250+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:24.584250+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23531520,"num_objects":5745,"num_object_clones":0,"num_object_copies":11490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11465,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15408","reported_seq":15549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.650708+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:24.650708+0000","last_peered":"2024-09-05T13:19:24.650708+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:24.650708+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691747,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12076,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":379605499,"num_objects":92679,"num_object_clones":0,"num_object_copies":185358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198508,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8485,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":979841024,"data_stored":979757007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628284,"kb_used_data":330020,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93743556,"statfs":{"total":96636764160,"available":95993401344,"internally_reserved":0,"allocated":337940480,"data_stored":337680923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764140,"kb_used_data":349340,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93607700,"statfs":{"total":96636764160,"available":95854284800,"internally_reserved":0,"allocated":357724160,"data_stored":357464598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627204,"kb_used_data":280360,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93744636,"statfs":{"total":96636764160,"available":95994507264,"internally_reserved":0,"allocated":287088640,"data_stored":286759972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356556800,"data_stored":356553187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286478336,"data_stored":286405617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336805888,"data_stored":336798203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:28.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:29.210 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:29.211 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:29.235 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-05T13:19:27.673602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":379605518,"num_objects":92681,"num_object_clones":0,"num_object_copies":185362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198510,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8486,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2019628,"kb_used_data":959720,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281095892,"statfs":{"total":289910292480,"available":287842193408,"internally_reserved":0,"allocated":982753280,"data_stored":981905493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1085262854},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,5,8,11,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19009536,"num_objects":-4641,"num_object_clones":0,"num_object_copies":-9282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4641,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004702"},"pg_stats":[{"pgid":"2.f","version":"45'15135","reported_seq":15210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.530977+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:24.530977+0000","last_peered":"2024-09-05T13:19:24.530977+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:24.530977+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15135,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1416}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15037","reported_seq":15143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.357965+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:23.357965+0000","last_peered":"2024-09-05T13:19:23.357965+0000","last_clean":"2024-09-05T13:19:23.357965+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:23.357965+0000","last_undegraded":"2024-09-05T13:19:23.357965+0000","last_fullsized":"2024-09-05T13:19:23.357965+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13047,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15309","reported_seq":15439,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.466291+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:23.466291+0000","last_peered":"2024-09-05T13:19:23.466291+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:23.466291+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109080,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13120,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1395}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15192","reported_seq":15263,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.467858+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:23.467858+0000","last_peered":"2024-09-05T13:19:23.467858+0000","last_clean":"2024-09-05T13:19:23.467858+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:23.467858+0000","last_undegraded":"2024-09-05T13:19:23.467858+0000","last_fullsized":"2024-09-05T13:19:23.467858+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15192,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15106","reported_seq":15241,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:24.637056+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:24.637056+0000","last_peered":"2024-09-05T13:19:24.637056+0000","last_clean":"2024-09-05T13:19:24.637056+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:24.637056+0000","last_undegraded":"2024-09-05T13:19:24.637056+0000","last_fullsized":"2024-09-05T13:19:24.637056+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23662592,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11646,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15290","reported_seq":15429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.598485+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:24.598485+0000","last_peered":"2024-09-05T13:19:24.598485+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:24.598485+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11745,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14970","reported_seq":23298,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.651933+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:24.651933+0000","last_peered":"2024-09-05T13:19:24.651933+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:24.651933+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175168,"num_objects":5658,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11289,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4093,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15293","reported_seq":15425,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:24.652636+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:24.652636+0000","last_peered":"2024-09-05T13:19:24.652636+0000","last_clean":"2024-09-05T13:19:24.652636+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:24.652636+0000","last_undegraded":"2024-09-05T13:19:24.652636+0000","last_fullsized":"2024-09-05T13:19:24.652636+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11903,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15228","reported_seq":15364,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.343604+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:23.343604+0000","last_peered":"2024-09-05T13:19:23.343604+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:23.343604+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12882,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15147","reported_seq":15286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.500374+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:24.500374+0000","last_peered":"2024-09-05T13:19:24.500374+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:24.500374+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11514,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15090","reported_seq":23992,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:23.424616+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:23.424616+0000","last_peered":"2024-09-05T13:19:23.424616+0000","last_clean":"2024-09-05T13:19:23.424616+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:23.424616+0000","last_undegraded":"2024-09-05T13:19:23.424616+0000","last_fullsized":"2024-09-05T13:19:23.424616+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12685,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14892","reported_seq":15015,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:23.442292+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:23.442292+0000","last_peered":"2024-09-05T13:19:23.442292+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:23.442292+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12401,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1303}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15108","reported_seq":15249,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.636981+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:24.636981+0000","last_peered":"2024-09-05T13:19:24.636981+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:24.636981+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11259,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15250","reported_seq":15438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.564133+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:24.564133+0000","last_peered":"2024-09-05T13:19:24.564133+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:24.564133+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11149,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15335","reported_seq":15475,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.584250+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:24.584250+0000","last_peered":"2024-09-05T13:19:24.584250+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:24.584250+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23531520,"num_objects":5745,"num_object_clones":0,"num_object_copies":11490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11465,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15408","reported_seq":15549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.650708+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:24.650708+0000","last_peered":"2024-09-05T13:19:24.650708+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:24.650708+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691747,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12076,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":379605499,"num_objects":92679,"num_object_clones":0,"num_object_copies":185358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198508,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8485,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":979841024,"data_stored":979757007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628284,"kb_used_data":330020,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93743556,"statfs":{"total":96636764160,"available":95993401344,"internally_reserved":0,"allocated":337940480,"data_stored":337680923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764140,"kb_used_data":349340,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93607700,"statfs":{"total":96636764160,"available":95854284800,"internally_reserved":0,"allocated":357724160,"data_stored":357464598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627204,"kb_used_data":280360,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93744636,"statfs":{"total":96636764160,"available":95994507264,"internally_reserved":0,"allocated":287088640,"data_stored":286759972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356556800,"data_stored":356553187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286478336,"data_stored":286405617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":336805888,"data_stored":336798203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:29.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:29.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'15135", 'reported_seq': 15210, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:24.530977+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:24.530977+0000', 'last_peered': '2024-09-05T13:19:24.530977+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:24.530977+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23908352, 'num_objects': 5837, 'num_object_clones': 0, 'num_object_copies': 11674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15135, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1416}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:29.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:29.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'15309", 'reported_seq': 15439, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:23.466291+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:19:23.466291+0000', 'last_peered': '2024-09-05T13:19:23.466291+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:23.466291+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24109080, 'num_objects': 5887, 'num_object_clones': 0, 'num_object_copies': 11774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5887, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13120, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1395}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:29.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'15290", 'reported_seq': 15429, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:24.598485+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:24.598485+0000', 'last_peered': '2024-09-05T13:19:24.598485+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:24.598485+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24023040, 'num_objects': 5865, 'num_object_clones': 0, 'num_object_copies': 11730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11745, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1414}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'14970", 'reported_seq': 23298, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:24.651933+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:19:24.651933+0000', 'last_peered': '2024-09-05T13:19:24.651933+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:19:24.651933+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23175168, 'num_objects': 5658, 'num_object_clones': 0, 'num_object_copies': 11316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 345, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11289, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4093, 'num_bytes_recovered': 15405056, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'15228", 'reported_seq': 15364, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:23.343604+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:23.343604+0000', 'last_peered': '2024-09-05T13:19:23.343604+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:23.343604+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24023040, 'num_objects': 5865, 'num_object_clones': 0, 'num_object_copies': 11730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12882, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1414}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'15147", 'reported_seq': 15286, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:24.500374+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:24.500374+0000', 'last_peered': '2024-09-05T13:19:24.500374+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:24.500374+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23605248, 'num_objects': 5763, 'num_object_clones': 0, 'num_object_copies': 11526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11514, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1309}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'14892", 'reported_seq': 15015, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:23.442292+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:23.442292+0000', 'last_peered': '2024-09-05T13:19:23.442292+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:23.442292+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23306240, 'num_objects': 5690, 'num_object_clones': 0, 'num_object_copies': 11380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12401, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1303}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:29.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:29.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'15108", 'reported_seq': 15249, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:24.636981+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:24.636981+0000', 'last_peered': '2024-09-05T13:19:24.636981+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:24.636981+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23633920, 'num_objects': 5770, 'num_object_clones': 0, 'num_object_copies': 11540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11259, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1351}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:29.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:29.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'15250", 'reported_seq': 15438, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:24.564133+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:24.564133+0000', 'last_peered': '2024-09-05T13:19:24.564133+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:24.564133+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23977984, 'num_objects': 5854, 'num_object_clones': 0, 'num_object_copies': 11708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11149, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1410}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:29.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:29.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'15335", 'reported_seq': 15475, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:24.584250+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:24.584250+0000', 'last_peered': '2024-09-05T13:19:24.584250+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:24.584250+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23531520, 'num_objects': 5745, 'num_object_clones': 0, 'num_object_copies': 11490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5745, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11465, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1179}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:29.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:29.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'15408", 'reported_seq': 15549, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:24.650708+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:24.650708+0000', 'last_peered': '2024-09-05T13:19:24.650708+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:24.650708+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23691747, 'num_objects': 5785, 'num_object_clones': 0, 'num_object_copies': 11570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5785, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12076, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:29.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:29.947 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:29.947 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45013/185362 objects degraded (24.284%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":496,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":92681,"data_bytes":379605518,"bytes_used":2068099072,"bytes_avail":287842193408,"bytes_total":289910292480,"degraded_objects":45013,"degraded_total":185362,"degraded_ratio":0.24283833795492066,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:29.973 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:29.973 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:30.544 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:30.546 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:30.569 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-05T13:19:29.674332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376099342,"num_objects":91825,"num_object_clones":0,"num_object_copies":183650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199366,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8486,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1999960,"kb_used_data":948388,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281115560,"statfs":{"total":289910292480,"available":287862333440,"internally_reserved":0,"allocated":971149312,"data_stored":970301525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1076743174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,6,8,14,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22515712,"num_objects":-5497,"num_object_clones":0,"num_object_copies":-10994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5497,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004610"},"pg_stats":[{"pgid":"2.f","version":"45'15135","reported_seq":15210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.530977+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:24.530977+0000","last_peered":"2024-09-05T13:19:24.530977+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:24.530977+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15135,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1416}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15175","reported_seq":15281,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:28.524761+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:28.524761+0000","last_peered":"2024-09-05T13:19:28.524761+0000","last_clean":"2024-09-05T13:19:28.524761+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:28.524761+0000","last_undegraded":"2024-09-05T13:19:28.524761+0000","last_fullsized":"2024-09-05T13:19:28.524761+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22851584,"num_objects":5579,"num_object_clones":0,"num_object_copies":11158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13185,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15430","reported_seq":15560,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:28.536553+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:28.536553+0000","last_peered":"2024-09-05T13:19:28.536553+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:28.536553+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23613464,"num_objects":5766,"num_object_clones":0,"num_object_copies":11532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13241,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15342","reported_seq":15413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:28.471125+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:28.471125+0000","last_peered":"2024-09-05T13:19:28.471125+0000","last_clean":"2024-09-05T13:19:28.471125+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:28.471125+0000","last_undegraded":"2024-09-05T13:19:28.471125+0000","last_fullsized":"2024-09-05T13:19:28.471125+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23281664,"num_objects":5684,"num_object_clones":0,"num_object_copies":11368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15342,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15106","reported_seq":15241,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:24.637056+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:24.637056+0000","last_peered":"2024-09-05T13:19:24.637056+0000","last_clean":"2024-09-05T13:19:24.637056+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:24.637056+0000","last_undegraded":"2024-09-05T13:19:24.637056+0000","last_fullsized":"2024-09-05T13:19:24.637056+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23662592,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11646,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15290","reported_seq":15429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.598485+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:24.598485+0000","last_peered":"2024-09-05T13:19:24.598485+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:24.598485+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11745,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14970","reported_seq":23298,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.651933+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:24.651933+0000","last_peered":"2024-09-05T13:19:24.651933+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:24.651933+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175168,"num_objects":5658,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11289,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4093,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15293","reported_seq":15425,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:24.652636+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:24.652636+0000","last_peered":"2024-09-05T13:19:24.652636+0000","last_clean":"2024-09-05T13:19:24.652636+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:24.652636+0000","last_undegraded":"2024-09-05T13:19:24.652636+0000","last_fullsized":"2024-09-05T13:19:24.652636+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11903,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15368","reported_seq":15502,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:28.487761+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:28.487761+0000","last_peered":"2024-09-05T13:19:28.487761+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:28.487761+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23449600,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13022,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15147","reported_seq":15286,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.500374+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:24.500374+0000","last_peered":"2024-09-05T13:19:24.500374+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:24.500374+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23605248,"num_objects":5763,"num_object_clones":0,"num_object_copies":11526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11514,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15243","reported_seq":24145,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:28.441089+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:28.441089+0000","last_peered":"2024-09-05T13:19:28.441089+0000","last_clean":"2024-09-05T13:19:28.441089+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:28.441089+0000","last_undegraded":"2024-09-05T13:19:28.441089+0000","last_fullsized":"2024-09-05T13:19:28.441089+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23216128,"num_objects":5668,"num_object_clones":0,"num_object_copies":11336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12838,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200982+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:16:09.200982+0000","last_peered":"2024-09-05T13:16:09.200982+0000","last_clean":"2024-09-05T13:16:09.200982+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:16:09.200982+0000","last_undegraded":"2024-09-05T13:16:09.200982+0000","last_fullsized":"2024-09-05T13:16:09.200982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15046","reported_seq":15170,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:28.473893+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:28.473893+0000","last_peered":"2024-09-05T13:19:28.473893+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:28.473893+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12555,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1149}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15108","reported_seq":15249,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.636981+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:24.636981+0000","last_peered":"2024-09-05T13:19:24.636981+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:24.636981+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11259,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.200907+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:16:09.200907+0000","last_peered":"2024-09-05T13:16:09.200907+0000","last_clean":"2024-09-05T13:16:09.200907+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:16:09.200907+0000","last_undegraded":"2024-09-05T13:16:09.200907+0000","last_fullsized":"2024-09-05T13:16:09.200907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15250","reported_seq":15438,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.564133+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:24.564133+0000","last_peered":"2024-09-05T13:19:24.564133+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:24.564133+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23977984,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11149,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15335","reported_seq":15475,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.584250+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:24.584250+0000","last_peered":"2024-09-05T13:19:24.584250+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:24.584250+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23531520,"num_objects":5745,"num_object_clones":0,"num_object_copies":11490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11465,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201488+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:16:09.201488+0000","last_peered":"2024-09-05T13:16:09.201488+0000","last_clean":"2024-09-05T13:16:09.201488+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:16:09.201488+0000","last_undegraded":"2024-09-05T13:16:09.201488+0000","last_fullsized":"2024-09-05T13:16:09.201488+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15408","reported_seq":15549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:24.650708+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:24.650708+0000","last_peered":"2024-09-05T13:19:24.650708+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:24.650708+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691747,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12076,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":376099323,"num_objects":91823,"num_object_clones":0,"num_object_copies":183646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199364,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8485,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":967946240,"data_stored":967862223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627800,"kb_used_data":325380,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93744040,"statfs":{"total":96636764160,"available":95993896960,"internally_reserved":0,"allocated":333189120,"data_stored":332929563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764140,"kb_used_data":349340,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93607700,"statfs":{"total":96636764160,"available":95854284800,"internally_reserved":0,"allocated":357724160,"data_stored":357464598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608020,"kb_used_data":273668,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93763820,"statfs":{"total":96636764160,"available":96014151680,"internally_reserved":0,"allocated":280236032,"data_stored":279907364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356556800,"data_stored":356553187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279404544,"data_stored":279331825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331984896,"data_stored":331977211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:33.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:34.157 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:34.187 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-05T13:19:33.675936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":366686734,"num_objects":89527,"num_object_clones":0,"num_object_copies":179054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201664,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8658,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1998120,"kb_used_data":929908,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281117400,"statfs":{"total":289910292480,"available":287864217600,"internally_reserved":0,"allocated":952225792,"data_stored":951378005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1093782534},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,3,6,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22413312,"num_objects":-5472,"num_object_clones":0,"num_object_copies":-10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5472,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004633"},"pg_stats":[{"pgid":"2.f","version":"45'15270","reported_seq":15346,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.639949+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:29.639949+0000","last_peered":"2024-09-05T13:19:29.639949+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:29.639949+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23355392,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15270,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15312","reported_seq":15418,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.428571+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:33.428571+0000","last_peered":"2024-09-05T13:19:33.428571+0000","last_clean":"2024-09-05T13:19:33.428571+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:33.428571+0000","last_undegraded":"2024-09-05T13:19:33.428571+0000","last_fullsized":"2024-09-05T13:19:33.428571+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22290432,"num_objects":5442,"num_object_clones":0,"num_object_copies":10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13322,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15572","reported_seq":15702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.538272+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:33.538272+0000","last_peered":"2024-09-05T13:19:33.538272+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:33.538272+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031832,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13383,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15482","reported_seq":15553,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.514068+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:33.514068+0000","last_peered":"2024-09-05T13:19:33.514068+0000","last_clean":"2024-09-05T13:19:33.514068+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:33.514068+0000","last_undegraded":"2024-09-05T13:19:33.514068+0000","last_fullsized":"2024-09-05T13:19:33.514068+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15482,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15247","reported_seq":15382,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:29.574295+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:29.574295+0000","last_peered":"2024-09-05T13:19:29.574295+0000","last_clean":"2024-09-05T13:19:29.574295+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:29.574295+0000","last_undegraded":"2024-09-05T13:19:29.574295+0000","last_fullsized":"2024-09-05T13:19:29.574295+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11787,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15449","reported_seq":15586,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.610229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:29.610229+0000","last_peered":"2024-09-05T13:19:29.610229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:29.610229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11904,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1255}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15112","reported_seq":23784,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.675426+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:29.675426+0000","last_peered":"2024-09-05T13:19:29.675426+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:29.675426+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22593536,"num_objects":5516,"num_object_clones":0,"num_object_copies":11032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11431,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4265,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15447","reported_seq":15580,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:29.519653+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:29.519653+0000","last_peered":"2024-09-05T13:19:29.519653+0000","last_clean":"2024-09-05T13:19:29.519653+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:29.519653+0000","last_undegraded":"2024-09-05T13:19:29.519653+0000","last_fullsized":"2024-09-05T13:19:29.519653+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23171072,"num_objects":5657,"num_object_clones":0,"num_object_copies":11314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12057,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15512","reported_seq":15648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.437911+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:33.437911+0000","last_peered":"2024-09-05T13:19:33.437911+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:33.437911+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1130}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15287","reported_seq":15426,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.562050+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:29.562050+0000","last_peered":"2024-09-05T13:19:29.562050+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:29.562050+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11654,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1169}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15381","reported_seq":24283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.495513+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:33.495513+0000","last_peered":"2024-09-05T13:19:33.495513+0000","last_clean":"2024-09-05T13:19:33.495513+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:33.495513+0000","last_undegraded":"2024-09-05T13:19:33.495513+0000","last_fullsized":"2024-09-05T13:19:33.495513+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22650880,"num_objects":5530,"num_object_clones":0,"num_object_copies":11060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":81,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359062+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:17:26.359062+0000","last_peered":"2024-09-05T13:17:26.359062+0000","last_clean":"2024-09-05T13:17:26.359062+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:17:26.359062+0000","last_undegraded":"2024-09-05T13:17:26.359062+0000","last_fullsized":"2024-09-05T13:17:26.359062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15187","reported_seq":15311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.513405+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:33.513405+0000","last_peered":"2024-09-05T13:19:33.513405+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:33.513405+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22097920,"num_objects":5395,"num_object_clones":0,"num_object_copies":10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15249","reported_seq":15390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.645090+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:29.645090+0000","last_peered":"2024-09-05T13:19:29.645090+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:29.645090+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056384,"num_objects":5629,"num_object_clones":0,"num_object_copies":11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11400,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359085+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:17:26.359085+0000","last_peered":"2024-09-05T13:17:26.359085+0000","last_clean":"2024-09-05T13:17:26.359085+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:17:26.359085+0000","last_undegraded":"2024-09-05T13:17:26.359085+0000","last_fullsized":"2024-09-05T13:17:26.359085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15409","reported_seq":15597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.651044+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:29.651044+0000","last_peered":"2024-09-05T13:19:29.651044+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:29.651044+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15483","reported_seq":15621,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.635711+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:29.635711+0000","last_peered":"2024-09-05T13:19:29.635711+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:29.635711+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22925312,"num_objects":5597,"num_object_clones":0,"num_object_copies":11194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11613,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1031}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359098+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:17:26.359098+0000","last_peered":"2024-09-05T13:17:26.359098+0000","last_clean":"2024-09-05T13:17:26.359098+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:17:26.359098+0000","last_undegraded":"2024-09-05T13:17:26.359098+0000","last_fullsized":"2024-09-05T13:17:26.359098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15545","reported_seq":15686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.668692+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:29.668692+0000","last_peered":"2024-09-05T13:19:29.668692+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:29.668692+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23130595,"num_objects":5648,"num_object_clones":0,"num_object_copies":11296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12213,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":366686715,"num_objects":89525,"num_object_clones":0,"num_object_copies":179050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201662,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8657,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":949321728,"data_stored":949237711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627448,"kb_used_data":320872,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93744392,"statfs":{"total":96636764160,"available":95994257408,"internally_reserved":0,"allocated":328572928,"data_stored":328313371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":761212,"kb_used_data":342248,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93610628,"statfs":{"total":96636764160,"available":95857283072,"internally_reserved":0,"allocated":350461952,"data_stored":350202390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609460,"kb_used_data":266788,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93762380,"statfs":{"total":96636764160,"available":96012677120,"internally_reserved":0,"allocated":273190912,"data_stored":272862244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":349474816,"data_stored":349471203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":272478208,"data_stored":272405489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327368704,"data_stored":327361019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:34.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:34.738 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:34.739 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:34.763 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-05T13:19:33.675936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":366686734,"num_objects":89527,"num_object_clones":0,"num_object_copies":179054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201664,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8658,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1998120,"kb_used_data":929908,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281117400,"statfs":{"total":289910292480,"available":287864217600,"internally_reserved":0,"allocated":952225792,"data_stored":951378005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1093782534},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,3,6,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22413312,"num_objects":-5472,"num_object_clones":0,"num_object_copies":-10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5472,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004633"},"pg_stats":[{"pgid":"2.f","version":"45'15270","reported_seq":15346,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.639949+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:29.639949+0000","last_peered":"2024-09-05T13:19:29.639949+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:29.639949+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23355392,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15270,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15312","reported_seq":15418,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.428571+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:33.428571+0000","last_peered":"2024-09-05T13:19:33.428571+0000","last_clean":"2024-09-05T13:19:33.428571+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:33.428571+0000","last_undegraded":"2024-09-05T13:19:33.428571+0000","last_fullsized":"2024-09-05T13:19:33.428571+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22290432,"num_objects":5442,"num_object_clones":0,"num_object_copies":10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13322,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15572","reported_seq":15702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.538272+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:33.538272+0000","last_peered":"2024-09-05T13:19:33.538272+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:33.538272+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031832,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13383,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15482","reported_seq":15553,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.514068+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:33.514068+0000","last_peered":"2024-09-05T13:19:33.514068+0000","last_clean":"2024-09-05T13:19:33.514068+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:33.514068+0000","last_undegraded":"2024-09-05T13:19:33.514068+0000","last_fullsized":"2024-09-05T13:19:33.514068+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15482,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15247","reported_seq":15382,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:29.574295+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:29.574295+0000","last_peered":"2024-09-05T13:19:29.574295+0000","last_clean":"2024-09-05T13:19:29.574295+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:29.574295+0000","last_undegraded":"2024-09-05T13:19:29.574295+0000","last_fullsized":"2024-09-05T13:19:29.574295+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11787,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15449","reported_seq":15586,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.610229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:29.610229+0000","last_peered":"2024-09-05T13:19:29.610229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:29.610229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11904,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1255}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15112","reported_seq":23784,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.675426+0000","last_change":"2024-09-05T13:16:07.632670+0000","last_active":"2024-09-05T13:19:29.675426+0000","last_peered":"2024-09-05T13:19:29.675426+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:29.675426+0000","last_undegraded":"2024-09-05T13:13:17.978312+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22593536,"num_objects":5516,"num_object_clones":0,"num_object_copies":11032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11431,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4265,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15447","reported_seq":15580,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:29.519653+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:29.519653+0000","last_peered":"2024-09-05T13:19:29.519653+0000","last_clean":"2024-09-05T13:19:29.519653+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:29.519653+0000","last_undegraded":"2024-09-05T13:19:29.519653+0000","last_fullsized":"2024-09-05T13:19:29.519653+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23171072,"num_objects":5657,"num_object_clones":0,"num_object_copies":11314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12057,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15512","reported_seq":15648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.437911+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:33.437911+0000","last_peered":"2024-09-05T13:19:33.437911+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:33.437911+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1130}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15287","reported_seq":15426,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.562050+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:29.562050+0000","last_peered":"2024-09-05T13:19:29.562050+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:29.562050+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11654,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1169}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15381","reported_seq":24283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.495513+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:33.495513+0000","last_peered":"2024-09-05T13:19:33.495513+0000","last_clean":"2024-09-05T13:19:33.495513+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:33.495513+0000","last_undegraded":"2024-09-05T13:19:33.495513+0000","last_fullsized":"2024-09-05T13:19:33.495513+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22650880,"num_objects":5530,"num_object_clones":0,"num_object_copies":11060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":81,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359062+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:17:26.359062+0000","last_peered":"2024-09-05T13:17:26.359062+0000","last_clean":"2024-09-05T13:17:26.359062+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:17:26.359062+0000","last_undegraded":"2024-09-05T13:17:26.359062+0000","last_fullsized":"2024-09-05T13:17:26.359062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15187","reported_seq":15311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.513405+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:33.513405+0000","last_peered":"2024-09-05T13:19:33.513405+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:33.513405+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22097920,"num_objects":5395,"num_object_clones":0,"num_object_copies":10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15249","reported_seq":15390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.645090+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:29.645090+0000","last_peered":"2024-09-05T13:19:29.645090+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:29.645090+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056384,"num_objects":5629,"num_object_clones":0,"num_object_copies":11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11400,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359085+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:17:26.359085+0000","last_peered":"2024-09-05T13:17:26.359085+0000","last_clean":"2024-09-05T13:17:26.359085+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:17:26.359085+0000","last_undegraded":"2024-09-05T13:17:26.359085+0000","last_fullsized":"2024-09-05T13:17:26.359085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15409","reported_seq":15597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.651044+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:29.651044+0000","last_peered":"2024-09-05T13:19:29.651044+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:29.651044+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15483","reported_seq":15621,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.635711+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:29.635711+0000","last_peered":"2024-09-05T13:19:29.635711+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:29.635711+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22925312,"num_objects":5597,"num_object_clones":0,"num_object_copies":11194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11613,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1031}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359098+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:17:26.359098+0000","last_peered":"2024-09-05T13:17:26.359098+0000","last_clean":"2024-09-05T13:17:26.359098+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:17:26.359098+0000","last_undegraded":"2024-09-05T13:17:26.359098+0000","last_fullsized":"2024-09-05T13:17:26.359098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15545","reported_seq":15686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:29.668692+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:29.668692+0000","last_peered":"2024-09-05T13:19:29.668692+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:29.668692+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23130595,"num_objects":5648,"num_object_clones":0,"num_object_copies":11296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12213,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":366686715,"num_objects":89525,"num_object_clones":0,"num_object_copies":179050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":201662,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8657,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":949321728,"data_stored":949237711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627448,"kb_used_data":320872,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93744392,"statfs":{"total":96636764160,"available":95994257408,"internally_reserved":0,"allocated":328572928,"data_stored":328313371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":761212,"kb_used_data":342248,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93610628,"statfs":{"total":96636764160,"available":95857283072,"internally_reserved":0,"allocated":350461952,"data_stored":350202390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609460,"kb_used_data":266788,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93762380,"statfs":{"total":96636764160,"available":96012677120,"internally_reserved":0,"allocated":273190912,"data_stored":272862244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":349474816,"data_stored":349471203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":272478208,"data_stored":272405489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327368704,"data_stored":327361019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:34.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:34.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'15270", 'reported_seq': 15346, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:29.639949+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:29.639949+0000', 'last_peered': '2024-09-05T13:19:29.639949+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:29.639949+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23355392, 'num_objects': 5702, 'num_object_clones': 0, 'num_object_copies': 11404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15270, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1281}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:34.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:34.764 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'15572", 'reported_seq': 15702, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:33.538272+0000', 'last_change': '2024-09-05T13:13:18.493512+0000', 'last_active': '2024-09-05T13:19:33.538272+0000', 'last_peered': '2024-09-05T13:19:33.538272+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:33.538272+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23031832, 'num_objects': 5624, 'num_object_clones': 0, 'num_object_copies': 11248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4493, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5624, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13383, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1132}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'15449", 'reported_seq': 15586, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:29.610229+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:29.610229+0000', 'last_peered': '2024-09-05T13:19:29.610229+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:29.610229+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23371776, 'num_objects': 5706, 'num_object_clones': 0, 'num_object_copies': 11412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11904, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1255}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'15112", 'reported_seq': 23784, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:29.675426+0000', 'last_change': '2024-09-05T13:16:07.632670+0000', 'last_active': '2024-09-05T13:19:29.675426+0000', 'last_peered': '2024-09-05T13:19:29.675426+0000', 'last_clean': '2024-09-05T13:13:15.884879+0000', 'last_became_active': '2024-09-05T13:13:18.442221+0000', 'last_became_peered': '2024-09-05T13:13:18.442221+0000', 'last_unstale': '2024-09-05T13:19:29.675426+0000', 'last_undegraded': '2024-09-05T13:13:17.978312+0000', 'last_fullsized': '2024-09-05T13:13:17.902227+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3737", 'last_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.539372+0000', 'objects_scrubbed': 3675, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1443, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22593536, 'num_objects': 5516, 'num_object_clones': 0, 'num_object_copies': 11032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 173, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11431, 'num_write_kb': 26544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4265, 'num_bytes_recovered': 15405056, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'15512", 'reported_seq': 15648, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:33.437911+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:33.437911+0000', 'last_peered': '2024-09-05T13:19:33.437911+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:33.437911+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22859776, 'num_objects': 5581, 'num_object_clones': 0, 'num_object_copies': 11162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13166, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'15287", 'reported_seq': 15426, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:29.562050+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:29.562050+0000', 'last_peered': '2024-09-05T13:19:29.562050+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:29.562050+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23031808, 'num_objects': 5623, 'num_object_clones': 0, 'num_object_copies': 11246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11654, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1169}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:34.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'15187", 'reported_seq': 15311, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:33.513405+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:33.513405+0000', 'last_peered': '2024-09-05T13:19:33.513405+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:33.513405+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22097920, 'num_objects': 5395, 'num_object_clones': 0, 'num_object_copies': 10790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12696, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1008}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:34.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:34.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'15249", 'reported_seq': 15390, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:29.645090+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:29.645090+0000', 'last_peered': '2024-09-05T13:19:29.645090+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:29.645090+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23056384, 'num_objects': 5629, 'num_object_clones': 0, 'num_object_copies': 11258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11400, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1210}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:34.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:34.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'15409", 'reported_seq': 15597, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:29.651044+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:29.651044+0000', 'last_peered': '2024-09-05T13:19:29.651044+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:29.651044+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23326720, 'num_objects': 5695, 'num_object_clones': 0, 'num_object_copies': 11390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11308, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1251}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:34.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:34.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'15483", 'reported_seq': 15621, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:29.635711+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:29.635711+0000', 'last_peered': '2024-09-05T13:19:29.635711+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:29.635711+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22925312, 'num_objects': 5597, 'num_object_clones': 0, 'num_object_copies': 11194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11613, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1031}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:34.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:34.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'15545", 'reported_seq': 15686, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:29.668692+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:29.668692+0000', 'last_peered': '2024-09-05T13:19:29.668692+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:29.668692+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23130595, 'num_objects': 5648, 'num_object_clones': 0, 'num_object_copies': 11296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5648, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12213, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1083}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:34.767 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:35.454 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44841/179054 objects degraded (25.043%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":502,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":89527,"data_bytes":366686734,"bytes_used":2046074880,"bytes_avail":287864217600,"bytes_total":289910292480,"degraded_objects":44841,"degraded_total":179054,"degraded_ratio":0.25043283031934499,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":347,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":455},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:35.476 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:35.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:35.944 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:35.956 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-05T13:19:35.676684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":360706574,"num_objects":88067,"num_object_clones":0,"num_object_copies":176134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44668,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88067,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203124,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8831,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982960,"kb_used_data":923060,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281132560,"statfs":{"total":289910292480,"available":287879741440,"internally_reserved":0,"allocated":945213440,"data_stored":944365653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1085262854},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,5,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18898944,"num_objects":-4614,"num_object_clones":0,"num_object_copies":-9228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4614,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003871"},"pg_stats":[{"pgid":"2.f","version":"45'15428","reported_seq":15504,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.579390+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:34.579390+0000","last_peered":"2024-09-05T13:19:34.579390+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:34.579390+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15312","reported_seq":15418,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.428571+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:33.428571+0000","last_peered":"2024-09-05T13:19:33.428571+0000","last_clean":"2024-09-05T13:19:33.428571+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:33.428571+0000","last_undegraded":"2024-09-05T13:19:33.428571+0000","last_fullsized":"2024-09-05T13:19:33.428571+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22290432,"num_objects":5442,"num_object_clones":0,"num_object_copies":10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13322,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15572","reported_seq":15702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.538272+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:33.538272+0000","last_peered":"2024-09-05T13:19:33.538272+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:33.538272+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031832,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13383,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15482","reported_seq":15553,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.514068+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:33.514068+0000","last_peered":"2024-09-05T13:19:33.514068+0000","last_clean":"2024-09-05T13:19:33.514068+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:33.514068+0000","last_undegraded":"2024-09-05T13:19:33.514068+0000","last_fullsized":"2024-09-05T13:19:33.514068+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15482,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15408","reported_seq":15543,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:34.692364+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:34.692364+0000","last_peered":"2024-09-05T13:19:34.692364+0000","last_clean":"2024-09-05T13:19:34.692364+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:34.692364+0000","last_undegraded":"2024-09-05T13:19:34.692364+0000","last_fullsized":"2024-09-05T13:19:34.692364+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11948,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15592","reported_seq":15728,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.685039+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:34.685039+0000","last_peered":"2024-09-05T13:19:34.685039+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:34.685039+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786048,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12047,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15237","reported_seq":24256,"reported_epoch":45,"state":"active+undersized+remapped","last_fresh":"2024-09-05T13:19:34.605753+0000","last_change":"2024-09-05T13:19:34.353330+0000","last_active":"2024-09-05T13:19:34.605753+0000","last_peered":"2024-09-05T13:19:34.605753+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:34.605753+0000","last_undegraded":"2024-09-05T13:19:34.605753+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22081536,"num_objects":5391,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":5391}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15586","reported_seq":15719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:34.644235+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:34.644235+0000","last_peered":"2024-09-05T13:19:34.644235+0000","last_clean":"2024-09-05T13:19:34.644235+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:34.644235+0000","last_undegraded":"2024-09-05T13:19:34.644235+0000","last_fullsized":"2024-09-05T13:19:34.644235+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12196,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15512","reported_seq":15648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.437911+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:33.437911+0000","last_peered":"2024-09-05T13:19:33.437911+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:33.437911+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1130}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15435","reported_seq":15572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.667048+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:34.667048+0000","last_peered":"2024-09-05T13:19:34.667048+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:34.667048+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11802,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1021}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15381","reported_seq":24283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.495513+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:33.495513+0000","last_peered":"2024-09-05T13:19:33.495513+0000","last_clean":"2024-09-05T13:19:33.495513+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:33.495513+0000","last_undegraded":"2024-09-05T13:19:33.495513+0000","last_fullsized":"2024-09-05T13:19:33.495513+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22650880,"num_objects":5530,"num_object_clones":0,"num_object_copies":11060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":81,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359062+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:17:26.359062+0000","last_peered":"2024-09-05T13:17:26.359062+0000","last_clean":"2024-09-05T13:17:26.359062+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:17:26.359062+0000","last_undegraded":"2024-09-05T13:17:26.359062+0000","last_fullsized":"2024-09-05T13:17:26.359062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15187","reported_seq":15311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.513405+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:33.513405+0000","last_peered":"2024-09-05T13:19:33.513405+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:33.513405+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22097920,"num_objects":5395,"num_object_clones":0,"num_object_copies":10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15417","reported_seq":15558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.608772+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:34.608772+0000","last_peered":"2024-09-05T13:19:34.608772+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:34.608772+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22368256,"num_objects":5461,"num_object_clones":0,"num_object_copies":10922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11568,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1042}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359085+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:17:26.359085+0000","last_peered":"2024-09-05T13:17:26.359085+0000","last_clean":"2024-09-05T13:17:26.359085+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:17:26.359085+0000","last_undegraded":"2024-09-05T13:17:26.359085+0000","last_fullsized":"2024-09-05T13:17:26.359085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15563","reported_seq":15752,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.666040+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:34.666040+0000","last_peered":"2024-09-05T13:19:34.666040+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:34.666040+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11462,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1097}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15609","reported_seq":15749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.659165+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:34.659165+0000","last_peered":"2024-09-05T13:19:34.659165+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:34.659165+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22409216,"num_objects":5471,"num_object_clones":0,"num_object_copies":10942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11739,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359098+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:17:26.359098+0000","last_peered":"2024-09-05T13:17:26.359098+0000","last_clean":"2024-09-05T13:17:26.359098+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:17:26.359098+0000","last_undegraded":"2024-09-05T13:17:26.359098+0000","last_fullsized":"2024-09-05T13:17:26.359098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15683","reported_seq":15824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.608218+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:34.608218+0000","last_peered":"2024-09-05T13:19:34.608218+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:34.608218+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22565347,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12351,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":360706555,"num_objects":88065,"num_object_clones":0,"num_object_copies":176130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44668,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203122,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8830,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942153728,"data_stored":942069711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627448,"kb_used_data":320872,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93744392,"statfs":{"total":96636764160,"available":95994257408,"internally_reserved":0,"allocated":328572928,"data_stored":328313371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746052,"kb_used_data":335400,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93625788,"statfs":{"total":96636764160,"available":95872806912,"internally_reserved":0,"allocated":343449600,"data_stored":343190038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609460,"kb_used_data":266788,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93762380,"statfs":{"total":96636764160,"available":96012677120,"internally_reserved":0,"allocated":273190912,"data_stored":272862244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342306816,"data_stored":342303203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":272478208,"data_stored":272405489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327368704,"data_stored":327361019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:38.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:39.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:39.515 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:39.540 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-05T13:19:37.677390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":360706574,"num_objects":88067,"num_object_clones":0,"num_object_copies":176134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44668,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88067,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203124,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8831,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1982960,"kb_used_data":923060,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281132560,"statfs":{"total":289910292480,"available":287879741440,"internally_reserved":0,"allocated":945213440,"data_stored":944365653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1085262854},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,5,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15392768,"num_objects":-3758,"num_object_clones":0,"num_object_copies":-7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3758,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003058"},"pg_stats":[{"pgid":"2.f","version":"45'15428","reported_seq":15504,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.579390+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:34.579390+0000","last_peered":"2024-09-05T13:19:34.579390+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:34.579390+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'15312","reported_seq":15418,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.428571+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:33.428571+0000","last_peered":"2024-09-05T13:19:33.428571+0000","last_clean":"2024-09-05T13:19:33.428571+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:33.428571+0000","last_undegraded":"2024-09-05T13:19:33.428571+0000","last_fullsized":"2024-09-05T13:19:33.428571+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22290432,"num_objects":5442,"num_object_clones":0,"num_object_copies":10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13322,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15572","reported_seq":15702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.538272+0000","last_change":"2024-09-05T13:13:18.493512+0000","last_active":"2024-09-05T13:19:33.538272+0000","last_peered":"2024-09-05T13:19:33.538272+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:33.538272+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23031832,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4493,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13383,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'15482","reported_seq":15553,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.514068+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:33.514068+0000","last_peered":"2024-09-05T13:19:33.514068+0000","last_clean":"2024-09-05T13:19:33.514068+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:33.514068+0000","last_undegraded":"2024-09-05T13:19:33.514068+0000","last_fullsized":"2024-09-05T13:19:33.514068+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15482,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15408","reported_seq":15543,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:34.692364+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:34.692364+0000","last_peered":"2024-09-05T13:19:34.692364+0000","last_clean":"2024-09-05T13:19:34.692364+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:34.692364+0000","last_undegraded":"2024-09-05T13:19:34.692364+0000","last_fullsized":"2024-09-05T13:19:34.692364+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11948,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15592","reported_seq":15728,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.685039+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:34.685039+0000","last_peered":"2024-09-05T13:19:34.685039+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:34.685039+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786048,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12047,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'15237","reported_seq":24256,"reported_epoch":45,"state":"active+undersized+remapped","last_fresh":"2024-09-05T13:19:34.605753+0000","last_change":"2024-09-05T13:19:34.353330+0000","last_active":"2024-09-05T13:19:34.605753+0000","last_peered":"2024-09-05T13:19:34.605753+0000","last_clean":"2024-09-05T13:13:15.884879+0000","last_became_active":"2024-09-05T13:13:18.442221+0000","last_became_peered":"2024-09-05T13:13:18.442221+0000","last_unstale":"2024-09-05T13:19:34.605753+0000","last_undegraded":"2024-09-05T13:19:34.605753+0000","last_fullsized":"2024-09-05T13:13:17.902227+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22081536,"num_objects":5391,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":5391}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'15586","reported_seq":15719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:34.644235+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:34.644235+0000","last_peered":"2024-09-05T13:19:34.644235+0000","last_clean":"2024-09-05T13:19:34.644235+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:34.644235+0000","last_undegraded":"2024-09-05T13:19:34.644235+0000","last_fullsized":"2024-09-05T13:19:34.644235+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12196,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:16.268656+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:17:16.268656+0000","last_peered":"2024-09-05T13:17:16.268656+0000","last_clean":"2024-09-05T13:17:16.268656+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:17:16.268656+0000","last_undegraded":"2024-09-05T13:17:16.268656+0000","last_fullsized":"2024-09-05T13:17:16.268656+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'15512","reported_seq":15648,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.437911+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:33.437911+0000","last_peered":"2024-09-05T13:19:33.437911+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:33.437911+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1130}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.913198+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:16:27.913198+0000","last_peered":"2024-09-05T13:16:27.913198+0000","last_clean":"2024-09-05T13:16:27.913198+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:16:27.913198+0000","last_undegraded":"2024-09-05T13:16:27.913198+0000","last_fullsized":"2024-09-05T13:16:27.913198+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15435","reported_seq":15572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.667048+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:34.667048+0000","last_peered":"2024-09-05T13:19:34.667048+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:34.667048+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11802,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1021}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'15381","reported_seq":24283,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:33.495513+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:33.495513+0000","last_peered":"2024-09-05T13:19:33.495513+0000","last_clean":"2024-09-05T13:19:33.495513+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:33.495513+0000","last_undegraded":"2024-09-05T13:19:33.495513+0000","last_fullsized":"2024-09-05T13:19:33.495513+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22650880,"num_objects":5530,"num_object_clones":0,"num_object_copies":11060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":81,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359062+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:17:26.359062+0000","last_peered":"2024-09-05T13:17:26.359062+0000","last_clean":"2024-09-05T13:17:26.359062+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:17:26.359062+0000","last_undegraded":"2024-09-05T13:17:26.359062+0000","last_fullsized":"2024-09-05T13:17:26.359062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'15187","reported_seq":15311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:33.513405+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:33.513405+0000","last_peered":"2024-09-05T13:19:33.513405+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:33.513405+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22097920,"num_objects":5395,"num_object_clones":0,"num_object_copies":10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1008}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15417","reported_seq":15558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.608772+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:34.608772+0000","last_peered":"2024-09-05T13:19:34.608772+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:34.608772+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22368256,"num_objects":5461,"num_object_clones":0,"num_object_copies":10922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11568,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1042}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359085+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:17:26.359085+0000","last_peered":"2024-09-05T13:17:26.359085+0000","last_clean":"2024-09-05T13:17:26.359085+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:17:26.359085+0000","last_undegraded":"2024-09-05T13:17:26.359085+0000","last_fullsized":"2024-09-05T13:17:26.359085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15563","reported_seq":15752,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.666040+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:34.666040+0000","last_peered":"2024-09-05T13:19:34.666040+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:34.666040+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11462,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1097}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:27.935036+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:16:27.935036+0000","last_peered":"2024-09-05T13:16:27.935036+0000","last_clean":"2024-09-05T13:16:27.935036+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:16:27.935036+0000","last_undegraded":"2024-09-05T13:16:27.935036+0000","last_fullsized":"2024-09-05T13:16:27.935036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15609","reported_seq":15749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.659165+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:34.659165+0000","last_peered":"2024-09-05T13:19:34.659165+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:34.659165+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22409216,"num_objects":5471,"num_object_clones":0,"num_object_copies":10942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11739,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:17:26.359098+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:17:26.359098+0000","last_peered":"2024-09-05T13:17:26.359098+0000","last_clean":"2024-09-05T13:17:26.359098+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:17:26.359098+0000","last_undegraded":"2024-09-05T13:17:26.359098+0000","last_fullsized":"2024-09-05T13:17:26.359098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:09.201262+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:16:09.201262+0000","last_peered":"2024-09-05T13:16:09.201262+0000","last_clean":"2024-09-05T13:16:09.201262+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:16:09.201262+0000","last_undegraded":"2024-09-05T13:16:09.201262+0000","last_fullsized":"2024-09-05T13:16:09.201262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15683","reported_seq":15824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.608218+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:34.608218+0000","last_peered":"2024-09-05T13:19:34.608218+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:34.608218+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22565347,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12351,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":360706555,"num_objects":88065,"num_object_clones":0,"num_object_copies":176130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44668,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203122,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8830,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":942153728,"data_stored":942069711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627448,"kb_used_data":320872,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93744392,"statfs":{"total":96636764160,"available":95994257408,"internally_reserved":0,"allocated":328572928,"data_stored":328313371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":313913005},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746052,"kb_used_data":335400,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93625788,"statfs":{"total":96636764160,"available":95872806912,"internally_reserved":0,"allocated":343449600,"data_stored":343190038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609460,"kb_used_data":266788,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93762380,"statfs":{"total":96636764160,"available":96012677120,"internally_reserved":0,"allocated":273190912,"data_stored":272862244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342306816,"data_stored":342303203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":272478208,"data_stored":272405489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327368704,"data_stored":327361019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:39.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:40.105 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:40.105 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:40.127 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-05T13:19:39.678161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":356979214,"num_objects":87157,"num_object_clones":0,"num_object_copies":174314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87157,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204034,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8948,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160617,"ondisk_log_size":160617,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1968144,"kb_used_data":912396,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281147376,"statfs":{"total":289910292480,"available":287894913024,"internally_reserved":0,"allocated":934293504,"data_stored":933428440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1081003014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,5,10,4,6,2,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13156352,"num_objects":-3212,"num_object_clones":0,"num_object_copies":-6424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002989"},"pg_stats":[{"pgid":"2.f","version":"45'15428","reported_seq":15504,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.579390+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:34.579390+0000","last_peered":"2024-09-05T13:19:34.579390+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:34.579390+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15453","reported_seq":15563,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:38.497646+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:38.497646+0000","last_peered":"2024-09-05T13:19:38.497646+0000","last_clean":"2024-09-05T13:19:38.497646+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:38.497646+0000","last_undegraded":"2024-09-05T13:19:38.497646+0000","last_fullsized":"2024-09-05T13:19:38.497646+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21712896,"num_objects":5301,"num_object_clones":0,"num_object_copies":10602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13463,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15714","reported_seq":16080,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:38.537929+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:38.537929+0000","last_peered":"2024-09-05T13:19:38.537929+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:38.537929+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22450200,"num_objects":5482,"num_object_clones":0,"num_object_copies":10964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4376,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13525,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1106}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15606","reported_seq":15681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:38.546731+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:38.546731+0000","last_peered":"2024-09-05T13:19:38.546731+0000","last_clean":"2024-09-05T13:19:38.546731+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:38.546731+0000","last_undegraded":"2024-09-05T13:19:38.546731+0000","last_fullsized":"2024-09-05T13:19:38.546731+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15606,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15408","reported_seq":15543,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:34.692364+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:34.692364+0000","last_peered":"2024-09-05T13:19:34.692364+0000","last_clean":"2024-09-05T13:19:34.692364+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:34.692364+0000","last_undegraded":"2024-09-05T13:19:34.692364+0000","last_fullsized":"2024-09-05T13:19:34.692364+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11948,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15592","reported_seq":15728,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.685039+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:34.685039+0000","last_peered":"2024-09-05T13:19:34.685039+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:34.685039+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786048,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12047,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15332","reported_seq":24363,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:38.098235+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:38.098235+0000","last_peered":"2024-09-05T13:19:38.098235+0000","last_clean":"2024-09-05T13:19:38.098235+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:38.098235+0000","last_undegraded":"2024-09-05T13:19:38.098235+0000","last_fullsized":"2024-09-05T13:19:38.098235+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":5296,"num_object_clones":0,"num_object_copies":10592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'15586","reported_seq":15719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:34.644235+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:34.644235+0000","last_peered":"2024-09-05T13:19:34.644235+0000","last_clean":"2024-09-05T13:19:34.644235+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:34.644235+0000","last_undegraded":"2024-09-05T13:19:34.644235+0000","last_fullsized":"2024-09-05T13:19:34.644235+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12196,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15633","reported_seq":15773,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:38.458209+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:38.458209+0000","last_peered":"2024-09-05T13:19:38.458209+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:38.458209+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22364160,"num_objects":5460,"num_object_clones":0,"num_object_copies":10920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13287,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1009}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15435","reported_seq":15572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.667048+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:34.667048+0000","last_peered":"2024-09-05T13:19:34.667048+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:34.667048+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11802,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1021}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15533","reported_seq":24439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:38.529234+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:38.529234+0000","last_peered":"2024-09-05T13:19:38.529234+0000","last_clean":"2024-09-05T13:19:38.529234+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:38.529234+0000","last_undegraded":"2024-09-05T13:19:38.529234+0000","last_fullsized":"2024-09-05T13:19:38.529234+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22028288,"num_objects":5378,"num_object_clones":0,"num_object_copies":10756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13128,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15322","reported_seq":15450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:38.544028+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:38.544028+0000","last_peered":"2024-09-05T13:19:38.544028+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:38.544028+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21544960,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12831,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15417","reported_seq":15558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.608772+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:34.608772+0000","last_peered":"2024-09-05T13:19:34.608772+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:34.608772+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22368256,"num_objects":5461,"num_object_clones":0,"num_object_copies":10922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11568,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1042}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15563","reported_seq":15752,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.666040+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:34.666040+0000","last_peered":"2024-09-05T13:19:34.666040+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:34.666040+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11462,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1097}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15609","reported_seq":15749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.659165+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:34.659165+0000","last_peered":"2024-09-05T13:19:34.659165+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:34.659165+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22409216,"num_objects":5471,"num_object_clones":0,"num_object_copies":10942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11739,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106451+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.106451+0000","last_peered":"2024-09-05T13:19:36.106451+0000","last_clean":"2024-09-05T13:19:36.106451+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.106451+0000","last_undegraded":"2024-09-05T13:19:36.106451+0000","last_fullsized":"2024-09-05T13:19:36.106451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15683","reported_seq":15824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.608218+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:34.608218+0000","last_peered":"2024-09-05T13:19:34.608218+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:34.608218+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22565347,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12351,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":356979195,"num_objects":87155,"num_object_clones":0,"num_object_copies":174310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204032,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8947,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":931000320,"data_stored":930919916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160614,"ondisk_log_size":160614,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627376,"kb_used_data":316632,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93744464,"statfs":{"total":96636764160,"available":95994331136,"internally_reserved":0,"allocated":324231168,"data_stored":323961166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746052,"kb_used_data":335400,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93625788,"statfs":{"total":96636764160,"available":95872806912,"internally_reserved":0,"allocated":343449600,"data_stored":343190038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594716,"kb_used_data":260364,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93777124,"statfs":{"total":96636764160,"available":96027774976,"internally_reserved":0,"allocated":266612736,"data_stored":266277236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,2,0,2,2,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342306816,"data_stored":342303203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":265912320,"data_stored":265843214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322781184,"data_stored":322773499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:40.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:40.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'15428", 'reported_seq': 15504, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:34.579390+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:34.579390+0000', 'last_peered': '2024-09-05T13:19:34.579390+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:34.579390+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22708224, 'num_objects': 5544, 'num_object_clones': 0, 'num_object_copies': 11088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15428, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:40.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:40.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15714", 'reported_seq': 16080, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:38.537929+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:19:38.537929+0000', 'last_peered': '2024-09-05T13:19:38.537929+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:38.537929+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22450200, 'num_objects': 5482, 'num_object_clones': 0, 'num_object_copies': 10964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4376, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5482, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13525, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 117, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1106}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:40.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:40.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'15592", 'reported_seq': 15728, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:34.685039+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:34.685039+0000', 'last_peered': '2024-09-05T13:19:34.685039+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:34.685039+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22786048, 'num_objects': 5563, 'num_object_clones': 0, 'num_object_copies': 11126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12047, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1112}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:40.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:40.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15633", 'reported_seq': 15773, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:38.458209+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:38.458209+0000', 'last_peered': '2024-09-05T13:19:38.458209+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:38.458209+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22364160, 'num_objects': 5460, 'num_object_clones': 0, 'num_object_copies': 10920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13287, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1009}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'15435", 'reported_seq': 15572, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:34.667048+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:34.667048+0000', 'last_peered': '2024-09-05T13:19:34.667048+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:34.667048+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22425600, 'num_objects': 5475, 'num_object_clones': 0, 'num_object_copies': 10950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5475, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11802, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1021}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15322", 'reported_seq': 15450, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:38.544028+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:38.544028+0000', 'last_peered': '2024-09-05T13:19:38.544028+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:38.544028+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21544960, 'num_objects': 5260, 'num_object_clones': 0, 'num_object_copies': 10520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12831, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 873}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'15417", 'reported_seq': 15558, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:34.608772+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:34.608772+0000', 'last_peered': '2024-09-05T13:19:34.608772+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:34.608772+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22368256, 'num_objects': 5461, 'num_object_clones': 0, 'num_object_copies': 10922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5461, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11568, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1042}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'15563", 'reported_seq': 15752, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:34.666040+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:34.666040+0000', 'last_peered': '2024-09-05T13:19:34.666040+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:34.666040+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22695936, 'num_objects': 5541, 'num_object_clones': 0, 'num_object_copies': 11082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11462, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1097}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'15609", 'reported_seq': 15749, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:34.659165+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:34.659165+0000', 'last_peered': '2024-09-05T13:19:34.659165+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:34.659165+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22409216, 'num_objects': 5471, 'num_object_clones': 0, 'num_object_copies': 10942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5471, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11739, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 905}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:40.130 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'15683", 'reported_seq': 15824, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:34.608218+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:34.608218+0000', 'last_peered': '2024-09-05T13:19:34.608218+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:34.608218+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22565347, 'num_objects': 5510, 'num_object_clones': 0, 'num_object_copies': 11020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12351, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 945}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:40.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:40.833 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:40.833 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44551/174314 objects degraded (25.558%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":507,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":87157,"data_bytes":356979214,"bytes_used":2015379456,"bytes_avail":287894913024,"bytes_total":289910292480,"degraded_objects":44551,"degraded_total":174314,"degraded_ratio":0.25557901258648186,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":290,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":401},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:40.857 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:40.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:41.404 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:41.406 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:41.431 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-05T13:19:39.678161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":356979214,"num_objects":87157,"num_object_clones":0,"num_object_copies":174314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87157,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204034,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8948,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160617,"ondisk_log_size":160617,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1968144,"kb_used_data":912396,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281147376,"statfs":{"total":289910292480,"available":287894913024,"internally_reserved":0,"allocated":934293504,"data_stored":933428440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1081003014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,4,5,10,4,6,2,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13156352,"num_objects":-3212,"num_object_clones":0,"num_object_copies":-6424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002989"},"pg_stats":[{"pgid":"2.f","version":"45'15428","reported_seq":15504,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.579390+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:34.579390+0000","last_peered":"2024-09-05T13:19:34.579390+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:34.579390+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22708224,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15453","reported_seq":15563,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:38.497646+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:38.497646+0000","last_peered":"2024-09-05T13:19:38.497646+0000","last_clean":"2024-09-05T13:19:38.497646+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:38.497646+0000","last_undegraded":"2024-09-05T13:19:38.497646+0000","last_fullsized":"2024-09-05T13:19:38.497646+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21712896,"num_objects":5301,"num_object_clones":0,"num_object_copies":10602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13463,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15714","reported_seq":16080,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:38.537929+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:38.537929+0000","last_peered":"2024-09-05T13:19:38.537929+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:38.537929+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22450200,"num_objects":5482,"num_object_clones":0,"num_object_copies":10964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4376,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13525,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1106}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15606","reported_seq":15681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:38.546731+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:38.546731+0000","last_peered":"2024-09-05T13:19:38.546731+0000","last_clean":"2024-09-05T13:19:38.546731+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:38.546731+0000","last_undegraded":"2024-09-05T13:19:38.546731+0000","last_fullsized":"2024-09-05T13:19:38.546731+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15606,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'15408","reported_seq":15543,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:34.692364+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:34.692364+0000","last_peered":"2024-09-05T13:19:34.692364+0000","last_clean":"2024-09-05T13:19:34.692364+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:34.692364+0000","last_undegraded":"2024-09-05T13:19:34.692364+0000","last_fullsized":"2024-09-05T13:19:34.692364+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11948,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'15592","reported_seq":15728,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.685039+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:34.685039+0000","last_peered":"2024-09-05T13:19:34.685039+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:34.685039+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786048,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12047,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15332","reported_seq":24363,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:38.098235+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:38.098235+0000","last_peered":"2024-09-05T13:19:38.098235+0000","last_clean":"2024-09-05T13:19:38.098235+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:38.098235+0000","last_undegraded":"2024-09-05T13:19:38.098235+0000","last_fullsized":"2024-09-05T13:19:38.098235+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21692416,"num_objects":5296,"num_object_clones":0,"num_object_copies":10592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'15586","reported_seq":15719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:19:34.644235+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:34.644235+0000","last_peered":"2024-09-05T13:19:34.644235+0000","last_clean":"2024-09-05T13:19:34.644235+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:34.644235+0000","last_undegraded":"2024-09-05T13:19:34.644235+0000","last_fullsized":"2024-09-05T13:19:34.644235+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22601728,"num_objects":5518,"num_object_clones":0,"num_object_copies":11036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12196,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15633","reported_seq":15773,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:38.458209+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:38.458209+0000","last_peered":"2024-09-05T13:19:38.458209+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:38.458209+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22364160,"num_objects":5460,"num_object_clones":0,"num_object_copies":10920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13287,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1009}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'15435","reported_seq":15572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.667048+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:34.667048+0000","last_peered":"2024-09-05T13:19:34.667048+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:34.667048+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11802,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1021}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-05T13:16:12.437804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:16:12.437804+0000","last_peered":"2024-09-05T13:16:12.437804+0000","last_clean":"2024-09-05T13:16:12.437804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:16:12.437804+0000","last_undegraded":"2024-09-05T13:16:12.437804+0000","last_fullsized":"2024-09-05T13:16:12.437804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15533","reported_seq":24439,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:38.529234+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:38.529234+0000","last_peered":"2024-09-05T13:19:38.529234+0000","last_clean":"2024-09-05T13:19:38.529234+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:38.529234+0000","last_undegraded":"2024-09-05T13:19:38.529234+0000","last_fullsized":"2024-09-05T13:19:38.529234+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22028288,"num_objects":5378,"num_object_clones":0,"num_object_copies":10756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13128,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15322","reported_seq":15450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:38.544028+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:38.544028+0000","last_peered":"2024-09-05T13:19:38.544028+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:38.544028+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21544960,"num_objects":5260,"num_object_clones":0,"num_object_copies":10520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12831,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'15417","reported_seq":15558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.608772+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:34.608772+0000","last_peered":"2024-09-05T13:19:34.608772+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:34.608772+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22368256,"num_objects":5461,"num_object_clones":0,"num_object_copies":10922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11568,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1042}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'15563","reported_seq":15752,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.666040+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:34.666040+0000","last_peered":"2024-09-05T13:19:34.666040+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:34.666040+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11462,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1097}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'15609","reported_seq":15749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.659165+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:34.659165+0000","last_peered":"2024-09-05T13:19:34.659165+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:34.659165+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22409216,"num_objects":5471,"num_object_clones":0,"num_object_copies":10942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11739,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106451+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.106451+0000","last_peered":"2024-09-05T13:19:36.106451+0000","last_clean":"2024-09-05T13:19:36.106451+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.106451+0000","last_undegraded":"2024-09-05T13:19:36.106451+0000","last_fullsized":"2024-09-05T13:19:36.106451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'15683","reported_seq":15824,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:34.608218+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:34.608218+0000","last_peered":"2024-09-05T13:19:34.608218+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:34.608218+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22565347,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12351,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":356979195,"num_objects":87155,"num_object_clones":0,"num_object_copies":174310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204032,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8947,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":931000320,"data_stored":930919916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160614,"ondisk_log_size":160614,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627376,"kb_used_data":316632,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93744464,"statfs":{"total":96636764160,"available":95994331136,"internally_reserved":0,"allocated":324231168,"data_stored":323961166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":318172845},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746052,"kb_used_data":335400,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93625788,"statfs":{"total":96636764160,"available":95872806912,"internally_reserved":0,"allocated":343449600,"data_stored":343190038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594716,"kb_used_data":260364,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93777124,"statfs":{"total":96636764160,"available":96027774976,"internally_reserved":0,"allocated":266612736,"data_stored":266277236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,2,0,2,2,5],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342306816,"data_stored":342303203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":265912320,"data_stored":265843214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322781184,"data_stored":322773499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:44.434 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:44.969 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:44.970 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:44.995 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-05T13:19:43.679671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":348156430,"num_objects":85003,"num_object_clones":0,"num_object_copies":170006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9021,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946020,"kb_used_data":894440,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281169500,"statfs":{"total":289910292480,"available":287917568000,"internally_reserved":0,"allocated":915906560,"data_stored":915031051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1076743174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,6,13,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12550144,"num_objects":-3064,"num_object_clones":0,"num_object_copies":-6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3064,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.589388"},"pg_stats":[{"pgid":"2.f","version":"47'15565","reported_seq":15643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.766360+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:39.766360+0000","last_peered":"2024-09-05T13:19:39.766360+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:39.766360+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15565,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":986}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15583","reported_seq":15693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.452282+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:43.452282+0000","last_peered":"2024-09-05T13:19:43.452282+0000","last_clean":"2024-09-05T13:19:43.452282+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:43.452282+0000","last_undegraded":"2024-09-05T13:19:43.452282+0000","last_fullsized":"2024-09-05T13:19:43.452282+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21180416,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13593,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15849","reported_seq":16364,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.549155+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:43.549155+0000","last_peered":"2024-09-05T13:19:43.549155+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:43.549155+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21897240,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13660,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1044}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15753","reported_seq":15827,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.561001+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:43.561001+0000","last_peered":"2024-09-05T13:19:43.561001+0000","last_clean":"2024-09-05T13:19:43.561001+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:43.561001+0000","last_undegraded":"2024-09-05T13:19:43.561001+0000","last_fullsized":"2024-09-05T13:19:43.561001+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21598208,"num_objects":5273,"num_object_clones":0,"num_object_copies":10546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15753,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15573","reported_seq":15712,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:39.704995+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:39.704995+0000","last_peered":"2024-09-05T13:19:39.704995+0000","last_clean":"2024-09-05T13:19:39.704995+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:39.704995+0000","last_undegraded":"2024-09-05T13:19:39.704995+0000","last_fullsized":"2024-09-05T13:19:39.704995+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749760,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12113,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15726","reported_seq":15869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.710073+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:39.710073+0000","last_peered":"2024-09-05T13:19:39.710073+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:39.710073+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22237184,"num_objects":5429,"num_object_clones":0,"num_object_copies":10858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12181,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15458","reported_seq":24490,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.053728+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:43.053728+0000","last_peered":"2024-09-05T13:19:43.053728+0000","last_clean":"2024-09-05T13:19:43.053728+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:43.053728+0000","last_undegraded":"2024-09-05T13:19:43.053728+0000","last_fullsized":"2024-09-05T13:19:43.053728+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11777,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'15733","reported_seq":15870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:39.738146+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:39.738146+0000","last_peered":"2024-09-05T13:19:39.738146+0000","last_clean":"2024-09-05T13:19:39.738146+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:39.738146+0000","last_undegraded":"2024-09-05T13:19:39.738146+0000","last_fullsized":"2024-09-05T13:19:39.738146+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21999616,"num_objects":5371,"num_object_clones":0,"num_object_copies":10742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12343,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15780","reported_seq":15920,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.415000+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:43.415000+0000","last_peered":"2024-09-05T13:19:43.415000+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:43.415000+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13434,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15582","reported_seq":15725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.649874+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:39.649874+0000","last_peered":"2024-09-05T13:19:39.649874+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:39.649874+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11949,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":874}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15650","reported_seq":24556,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.532356+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:43.532356+0000","last_peered":"2024-09-05T13:19:43.532356+0000","last_clean":"2024-09-05T13:19:43.532356+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:43.532356+0000","last_undegraded":"2024-09-05T13:19:43.532356+0000","last_fullsized":"2024-09-05T13:19:43.532356+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21549056,"num_objects":5261,"num_object_clones":0,"num_object_copies":10522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13245,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15461","reported_seq":15589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.474821+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:43.474821+0000","last_peered":"2024-09-05T13:19:43.474821+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:43.474821+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12970,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15536","reported_seq":15682,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.783969+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:39.783969+0000","last_peered":"2024-09-05T13:19:39.783969+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:39.783969+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21880832,"num_objects":5342,"num_object_clones":0,"num_object_copies":10684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11687,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15687","reported_seq":15880,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.645896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:39.645896+0000","last_peered":"2024-09-05T13:19:39.645896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:39.645896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22188032,"num_objects":5417,"num_object_clones":0,"num_object_copies":10834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11586,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15731","reported_seq":15875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.737741+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:39.737741+0000","last_peered":"2024-09-05T13:19:39.737741+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:39.737741+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21909504,"num_objects":5349,"num_object_clones":0,"num_object_copies":10698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11861,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":783}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106451+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.106451+0000","last_peered":"2024-09-05T13:19:36.106451+0000","last_clean":"2024-09-05T13:19:36.106451+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.106451+0000","last_undegraded":"2024-09-05T13:19:36.106451+0000","last_fullsized":"2024-09-05T13:19:36.106451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15801","reported_seq":15946,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.655982+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:39.655982+0000","last_peered":"2024-09-05T13:19:39.655982+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:39.655982+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22082019,"num_objects":5392,"num_object_clones":0,"num_object_copies":10784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12469,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":827}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":348156411,"num_objects":85001,"num_object_clones":0,"num_object_copies":170002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9020,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":912793600,"data_stored":912713196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610268,"kb_used_data":312004,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93761572,"statfs":{"total":96636764160,"available":96011849728,"internally_reserved":0,"allocated":319492096,"data_stored":319222094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743736,"kb_used_data":328936,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93628104,"statfs":{"total":96636764160,"available":95875178496,"internally_reserved":0,"allocated":336830464,"data_stored":336560457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592016,"kb_used_data":253500,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93779824,"statfs":{"total":96636764160,"available":96030539776,"internally_reserved":0,"allocated":259584000,"data_stored":259248500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335683584,"data_stored":335679971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":258899968,"data_stored":258830862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318210048,"data_stored":318202363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:44.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:45.549 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:45.549 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:45.570 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-05T13:19:43.679671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":348156430,"num_objects":85003,"num_object_clones":0,"num_object_copies":170006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9021,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946020,"kb_used_data":894440,"kb_used_omap":12,"kb_used_meta":1051507,"kb_avail":281169500,"statfs":{"total":289910292480,"available":287917568000,"internally_reserved":0,"allocated":915906560,"data_stored":915031051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1076743174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,6,13,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12550144,"num_objects":-3064,"num_object_clones":0,"num_object_copies":-6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3064,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.589388"},"pg_stats":[{"pgid":"2.f","version":"47'15565","reported_seq":15643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.766360+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:39.766360+0000","last_peered":"2024-09-05T13:19:39.766360+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:39.766360+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15565,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":986}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15583","reported_seq":15693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.452282+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:43.452282+0000","last_peered":"2024-09-05T13:19:43.452282+0000","last_clean":"2024-09-05T13:19:43.452282+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:43.452282+0000","last_undegraded":"2024-09-05T13:19:43.452282+0000","last_fullsized":"2024-09-05T13:19:43.452282+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21180416,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13593,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15849","reported_seq":16364,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.549155+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:43.549155+0000","last_peered":"2024-09-05T13:19:43.549155+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:43.549155+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21897240,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13660,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1044}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15753","reported_seq":15827,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.561001+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:43.561001+0000","last_peered":"2024-09-05T13:19:43.561001+0000","last_clean":"2024-09-05T13:19:43.561001+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:43.561001+0000","last_undegraded":"2024-09-05T13:19:43.561001+0000","last_fullsized":"2024-09-05T13:19:43.561001+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21598208,"num_objects":5273,"num_object_clones":0,"num_object_copies":10546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15753,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15573","reported_seq":15712,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:39.704995+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:39.704995+0000","last_peered":"2024-09-05T13:19:39.704995+0000","last_clean":"2024-09-05T13:19:39.704995+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:39.704995+0000","last_undegraded":"2024-09-05T13:19:39.704995+0000","last_fullsized":"2024-09-05T13:19:39.704995+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749760,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12113,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15726","reported_seq":15869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.710073+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:39.710073+0000","last_peered":"2024-09-05T13:19:39.710073+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:39.710073+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22237184,"num_objects":5429,"num_object_clones":0,"num_object_copies":10858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12181,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":978}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15458","reported_seq":24490,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.053728+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:43.053728+0000","last_peered":"2024-09-05T13:19:43.053728+0000","last_clean":"2024-09-05T13:19:43.053728+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:43.053728+0000","last_undegraded":"2024-09-05T13:19:43.053728+0000","last_fullsized":"2024-09-05T13:19:43.053728+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11777,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'15733","reported_seq":15870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:39.738146+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:39.738146+0000","last_peered":"2024-09-05T13:19:39.738146+0000","last_clean":"2024-09-05T13:19:39.738146+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:39.738146+0000","last_undegraded":"2024-09-05T13:19:39.738146+0000","last_fullsized":"2024-09-05T13:19:39.738146+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21999616,"num_objects":5371,"num_object_clones":0,"num_object_copies":10742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12343,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15780","reported_seq":15920,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.415000+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:43.415000+0000","last_peered":"2024-09-05T13:19:43.415000+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:43.415000+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13434,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15582","reported_seq":15725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.649874+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:39.649874+0000","last_peered":"2024-09-05T13:19:39.649874+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:39.649874+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823488,"num_objects":5328,"num_object_clones":0,"num_object_copies":10656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11949,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":874}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15650","reported_seq":24556,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.532356+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:43.532356+0000","last_peered":"2024-09-05T13:19:43.532356+0000","last_clean":"2024-09-05T13:19:43.532356+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:43.532356+0000","last_undegraded":"2024-09-05T13:19:43.532356+0000","last_fullsized":"2024-09-05T13:19:43.532356+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21549056,"num_objects":5261,"num_object_clones":0,"num_object_copies":10522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13245,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15461","reported_seq":15589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.474821+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:43.474821+0000","last_peered":"2024-09-05T13:19:43.474821+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:43.474821+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12970,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15536","reported_seq":15682,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.783969+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:39.783969+0000","last_peered":"2024-09-05T13:19:39.783969+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:39.783969+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21880832,"num_objects":5342,"num_object_clones":0,"num_object_copies":10684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11687,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15687","reported_seq":15880,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.645896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:39.645896+0000","last_peered":"2024-09-05T13:19:39.645896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:39.645896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22188032,"num_objects":5417,"num_object_clones":0,"num_object_copies":10834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11586,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15731","reported_seq":15875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.737741+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:39.737741+0000","last_peered":"2024-09-05T13:19:39.737741+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:39.737741+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21909504,"num_objects":5349,"num_object_clones":0,"num_object_copies":10698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11861,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":783}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106451+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.106451+0000","last_peered":"2024-09-05T13:19:36.106451+0000","last_clean":"2024-09-05T13:19:36.106451+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.106451+0000","last_undegraded":"2024-09-05T13:19:36.106451+0000","last_fullsized":"2024-09-05T13:19:36.106451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15801","reported_seq":15946,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:39.655982+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:39.655982+0000","last_peered":"2024-09-05T13:19:39.655982+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:39.655982+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22082019,"num_objects":5392,"num_object_clones":0,"num_object_copies":10784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12469,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":827}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":348156411,"num_objects":85001,"num_object_clones":0,"num_object_copies":170002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9020,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":912793600,"data_stored":912713196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610268,"kb_used_data":312004,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93761572,"statfs":{"total":96636764160,"available":96011849728,"internally_reserved":0,"allocated":319492096,"data_stored":319222094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743736,"kb_used_data":328936,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93628104,"statfs":{"total":96636764160,"available":95875178496,"internally_reserved":0,"allocated":336830464,"data_stored":336560457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592016,"kb_used_data":253500,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93779824,"statfs":{"total":96636764160,"available":96030539776,"internally_reserved":0,"allocated":259584000,"data_stored":259248500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335683584,"data_stored":335679971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":258899968,"data_stored":258830862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318210048,"data_stored":318202363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:45.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:45.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15565", 'reported_seq': 15643, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:39.766360+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:39.766360+0000', 'last_peered': '2024-09-05T13:19:39.766360+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:39.766360+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22147072, 'num_objects': 5407, 'num_object_clones': 0, 'num_object_copies': 10814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15565, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 986}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:45.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:45.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15849", 'reported_seq': 16364, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:43.549155+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:19:43.549155+0000', 'last_peered': '2024-09-05T13:19:43.549155+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:43.549155+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21897240, 'num_objects': 5347, 'num_object_clones': 0, 'num_object_copies': 10694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4303, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5347, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13660, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 190, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1044}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:45.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:45.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15726", 'reported_seq': 15869, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:39.710073+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:39.710073+0000', 'last_peered': '2024-09-05T13:19:39.710073+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:39.710073+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22237184, 'num_objects': 5429, 'num_object_clones': 0, 'num_object_copies': 10858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12181, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 978}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:45.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:45.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15780", 'reported_seq': 15920, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:43.415000+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:43.415000+0000', 'last_peered': '2024-09-05T13:19:43.415000+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:43.415000+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21762048, 'num_objects': 5313, 'num_object_clones': 0, 'num_object_copies': 10626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13434, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 862}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15582", 'reported_seq': 15725, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:39.649874+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:39.649874+0000', 'last_peered': '2024-09-05T13:19:39.649874+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:39.649874+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21823488, 'num_objects': 5328, 'num_object_clones': 0, 'num_object_copies': 10656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11949, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 874}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15461", 'reported_seq': 15589, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:43.474821+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:43.474821+0000', 'last_peered': '2024-09-05T13:19:43.474821+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:43.474821+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20975616, 'num_objects': 5121, 'num_object_clones': 0, 'num_object_copies': 10242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12970, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 734}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15536", 'reported_seq': 15682, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:39.783969+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:39.783969+0000', 'last_peered': '2024-09-05T13:19:39.783969+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:39.783969+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21880832, 'num_objects': 5342, 'num_object_clones': 0, 'num_object_copies': 10684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11687, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 923}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15687", 'reported_seq': 15880, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:39.645896+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:39.645896+0000', 'last_peered': '2024-09-05T13:19:39.645896+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:39.645896+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22188032, 'num_objects': 5417, 'num_object_clones': 0, 'num_object_copies': 10834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5417, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11586, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 973}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:45.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:45.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15731", 'reported_seq': 15875, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:39.737741+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:39.737741+0000', 'last_peered': '2024-09-05T13:19:39.737741+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:39.737741+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21909504, 'num_objects': 5349, 'num_object_clones': 0, 'num_object_copies': 10698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5349, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11861, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 783}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:45.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:45.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15801", 'reported_seq': 15946, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:39.655982+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:39.655982+0000', 'last_peered': '2024-09-05T13:19:39.655982+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:39.655982+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22082019, 'num_objects': 5392, 'num_object_clones': 0, 'num_object_copies': 10784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12469, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 827}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:45.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:46.279 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:46.280 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44478/167342 objects degraded (26.579%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":513,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":83671,"data_bytes":342700558,"bytes_used":1994137600,"bytes_avail":287916154880,"bytes_total":289910292480,"degraded_objects":44478,"degraded_total":167342,"degraded_ratio":0.26579101480799799,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":439},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:46.303 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:46.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:46.849 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:46.850 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:46.881 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-05T13:19:45.680466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":342700558,"num_objects":83671,"num_object_clones":0,"num_object_copies":167342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207520,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9021,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1947400,"kb_used_data":887500,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281168120,"statfs":{"total":289910292480,"available":287916154880,"internally_reserved":0,"allocated":908800000,"data_stored":907924491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1085262854},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,5,11,8,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18006016,"num_objects":-4396,"num_object_clones":0,"num_object_copies":-8792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4396,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003781"},"pg_stats":[{"pgid":"2.f","version":"47'15703","reported_seq":15783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.722070+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:44.722070+0000","last_peered":"2024-09-05T13:19:44.722070+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:44.722070+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21581824,"num_objects":5269,"num_object_clones":0,"num_object_copies":10538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15703,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":848}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15583","reported_seq":15693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.452282+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:43.452282+0000","last_peered":"2024-09-05T13:19:43.452282+0000","last_clean":"2024-09-05T13:19:43.452282+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:43.452282+0000","last_undegraded":"2024-09-05T13:19:43.452282+0000","last_fullsized":"2024-09-05T13:19:43.452282+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21180416,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13593,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15849","reported_seq":16364,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.549155+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:43.549155+0000","last_peered":"2024-09-05T13:19:43.549155+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:43.549155+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21897240,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13660,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1044}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15753","reported_seq":15827,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.561001+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:43.561001+0000","last_peered":"2024-09-05T13:19:43.561001+0000","last_clean":"2024-09-05T13:19:43.561001+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:43.561001+0000","last_undegraded":"2024-09-05T13:19:43.561001+0000","last_fullsized":"2024-09-05T13:19:43.561001+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21598208,"num_objects":5273,"num_object_clones":0,"num_object_copies":10546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15753,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15718","reported_seq":15858,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:44.647931+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:44.647931+0000","last_peered":"2024-09-05T13:19:44.647931+0000","last_clean":"2024-09-05T13:19:44.647931+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:44.647931+0000","last_undegraded":"2024-09-05T13:19:44.647931+0000","last_fullsized":"2024-09-05T13:19:44.647931+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12258,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15865","reported_seq":16007,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.779396+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:44.779396+0000","last_peered":"2024-09-05T13:19:44.779396+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:44.779396+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21667840,"num_objects":5290,"num_object_clones":0,"num_object_copies":10580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12320,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":839}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15458","reported_seq":24490,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.053728+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:43.053728+0000","last_peered":"2024-09-05T13:19:43.053728+0000","last_clean":"2024-09-05T13:19:43.053728+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:43.053728+0000","last_undegraded":"2024-09-05T13:19:43.053728+0000","last_fullsized":"2024-09-05T13:19:43.053728+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11777,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'15879","reported_seq":16016,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:44.748172+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:44.748172+0000","last_peered":"2024-09-05T13:19:44.748172+0000","last_clean":"2024-09-05T13:19:44.748172+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:44.748172+0000","last_undegraded":"2024-09-05T13:19:44.748172+0000","last_fullsized":"2024-09-05T13:19:44.748172+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12489,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15780","reported_seq":15920,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.415000+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:43.415000+0000","last_peered":"2024-09-05T13:19:43.415000+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:43.415000+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21762048,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13434,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":862}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15726","reported_seq":15866,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.793960+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:44.793960+0000","last_peered":"2024-09-05T13:19:44.793960+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:44.793960+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12093,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15650","reported_seq":24556,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:43.532356+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:43.532356+0000","last_peered":"2024-09-05T13:19:43.532356+0000","last_clean":"2024-09-05T13:19:43.532356+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:43.532356+0000","last_undegraded":"2024-09-05T13:19:43.532356+0000","last_fullsized":"2024-09-05T13:19:43.532356+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21549056,"num_objects":5261,"num_object_clones":0,"num_object_copies":10522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13245,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15461","reported_seq":15589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:43.474821+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:43.474821+0000","last_peered":"2024-09-05T13:19:43.474821+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:43.474821+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20975616,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12970,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15688","reported_seq":15834,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.789308+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:44.789308+0000","last_peered":"2024-09-05T13:19:44.789308+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:44.789308+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21258240,"num_objects":5190,"num_object_clones":0,"num_object_copies":10380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11839,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":771}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15847","reported_seq":16040,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.804696+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:44.804696+0000","last_peered":"2024-09-05T13:19:44.804696+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:44.804696+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21532672,"num_objects":5257,"num_object_clones":0,"num_object_copies":10514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11746,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":813}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15884","reported_seq":16028,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.809577+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:44.809577+0000","last_peered":"2024-09-05T13:19:44.809577+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:44.809577+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12014,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":630}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106451+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.106451+0000","last_peered":"2024-09-05T13:19:36.106451+0000","last_clean":"2024-09-05T13:19:36.106451+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.106451+0000","last_undegraded":"2024-09-05T13:19:36.106451+0000","last_fullsized":"2024-09-05T13:19:36.106451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15956","reported_seq":16101,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.815082+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:44.815082+0000","last_peered":"2024-09-05T13:19:44.815082+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:44.815082+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21447139,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12624,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":342700539,"num_objects":83669,"num_object_clones":0,"num_object_copies":167338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44478,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83669,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207518,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9020,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":905699328,"data_stored":905618924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610268,"kb_used_data":312004,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93761572,"statfs":{"total":96636764160,"available":96011849728,"internally_reserved":0,"allocated":319492096,"data_stored":319222094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":305393325},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745116,"kb_used_data":321996,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93626724,"statfs":{"total":96636764160,"available":95873765376,"internally_reserved":0,"allocated":329723904,"data_stored":329453897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592016,"kb_used_data":253500,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93779824,"statfs":{"total":96636764160,"available":96030539776,"internally_reserved":0,"allocated":259584000,"data_stored":259248500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328589312,"data_stored":328585699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":258899968,"data_stored":258830862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318210048,"data_stored":318202363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:49.884 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:50.466 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:50.493 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-05T13:19:49.681964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338473486,"num_objects":82639,"num_object_clones":0,"num_object_copies":165278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208552,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9076,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948164,"kb_used_data":875796,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281167356,"statfs":{"total":289910292480,"available":287915372544,"internally_reserved":0,"allocated":896815104,"data_stored":895939595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1098042374},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,6,7,6,11,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22233088,"num_objects":-5428,"num_object_clones":0,"num_object_copies":-10856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5428,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004573"},"pg_stats":[{"pgid":"2.f","version":"47'15703","reported_seq":15783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.722070+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:44.722070+0000","last_peered":"2024-09-05T13:19:44.722070+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:44.722070+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21581824,"num_objects":5269,"num_object_clones":0,"num_object_copies":10538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15703,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":848}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15714","reported_seq":15824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.520897+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:48.520897+0000","last_peered":"2024-09-05T13:19:48.520897+0000","last_clean":"2024-09-05T13:19:48.520897+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:48.520897+0000","last_undegraded":"2024-09-05T13:19:48.520897+0000","last_fullsized":"2024-09-05T13:19:48.520897+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13724,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16011","reported_seq":16636,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.549789+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:48.549789+0000","last_peered":"2024-09-05T13:19:48.549789+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:48.549789+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233688,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13822,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15888","reported_seq":15963,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.533541+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:48.533541+0000","last_peered":"2024-09-05T13:19:48.533541+0000","last_clean":"2024-09-05T13:19:48.533541+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:48.533541+0000","last_undegraded":"2024-09-05T13:19:48.533541+0000","last_fullsized":"2024-09-05T13:19:48.533541+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21045248,"num_objects":5138,"num_object_clones":0,"num_object_copies":10276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15888,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15718","reported_seq":15858,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:44.647931+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:44.647931+0000","last_peered":"2024-09-05T13:19:44.647931+0000","last_clean":"2024-09-05T13:19:44.647931+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:44.647931+0000","last_undegraded":"2024-09-05T13:19:44.647931+0000","last_fullsized":"2024-09-05T13:19:44.647931+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12258,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15865","reported_seq":16007,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.779396+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:44.779396+0000","last_peered":"2024-09-05T13:19:44.779396+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:44.779396+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21667840,"num_objects":5290,"num_object_clones":0,"num_object_copies":10580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12320,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":839}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15611","reported_seq":24643,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.324603+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:48.324603+0000","last_peered":"2024-09-05T13:19:48.324603+0000","last_clean":"2024-09-05T13:19:48.324603+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:48.324603+0000","last_undegraded":"2024-09-05T13:19:48.324603+0000","last_fullsized":"2024-09-05T13:19:48.324603+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20549632,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11930,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'15879","reported_seq":16016,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:44.748172+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:44.748172+0000","last_peered":"2024-09-05T13:19:44.748172+0000","last_clean":"2024-09-05T13:19:44.748172+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:44.748172+0000","last_undegraded":"2024-09-05T13:19:44.748172+0000","last_fullsized":"2024-09-05T13:19:44.748172+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12489,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15928","reported_seq":16068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.565445+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:48.565445+0000","last_peered":"2024-09-05T13:19:48.565445+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:48.565445+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13582,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":714}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15726","reported_seq":15866,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.793960+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:44.793960+0000","last_peered":"2024-09-05T13:19:44.793960+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:44.793960+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12093,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15793","reported_seq":24699,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.541658+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:48.541658+0000","last_peered":"2024-09-05T13:19:48.541658+0000","last_clean":"2024-09-05T13:19:48.541658+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:48.541658+0000","last_undegraded":"2024-09-05T13:19:48.541658+0000","last_fullsized":"2024-09-05T13:19:48.541658+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15621","reported_seq":15749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.575263+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:48.575263+0000","last_peered":"2024-09-05T13:19:48.575263+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:48.575263+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20320256,"num_objects":4961,"num_object_clones":0,"num_object_copies":9922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13130,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":574}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15688","reported_seq":15834,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.789308+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:44.789308+0000","last_peered":"2024-09-05T13:19:44.789308+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:44.789308+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21258240,"num_objects":5190,"num_object_clones":0,"num_object_copies":10380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11839,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":771}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15847","reported_seq":16040,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.804696+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:44.804696+0000","last_peered":"2024-09-05T13:19:44.804696+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:44.804696+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21532672,"num_objects":5257,"num_object_clones":0,"num_object_copies":10514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11746,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":813}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15884","reported_seq":16028,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.809577+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:44.809577+0000","last_peered":"2024-09-05T13:19:44.809577+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:44.809577+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12014,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":630}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15956","reported_seq":16101,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.815082+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:44.815082+0000","last_peered":"2024-09-05T13:19:44.815082+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:44.815082+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21447139,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12624,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338473467,"num_objects":82637,"num_object_clones":0,"num_object_copies":165274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208550,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9075,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":893681664,"data_stored":893601260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609732,"kb_used_data":307316,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93762108,"statfs":{"total":96636764160,"available":96012398592,"internally_reserved":0,"allocated":314691584,"data_stored":314421582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745116,"kb_used_data":321996,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93626724,"statfs":{"total":96636764160,"available":95873765376,"internally_reserved":0,"allocated":329723904,"data_stored":329453897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593316,"kb_used_data":246484,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93778524,"statfs":{"total":96636764160,"available":96029208576,"internally_reserved":0,"allocated":252399616,"data_stored":252064116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,1,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328589312,"data_stored":328585699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":251654144,"data_stored":251585038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313438208,"data_stored":313430523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:50.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:51.034 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:51.037 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:51.060 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-05T13:19:49.681964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338473486,"num_objects":82639,"num_object_clones":0,"num_object_copies":165278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208552,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9076,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1948164,"kb_used_data":875796,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281167356,"statfs":{"total":289910292480,"available":287915372544,"internally_reserved":0,"allocated":896815104,"data_stored":895939595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1098042374},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,6,7,6,11,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22233088,"num_objects":-5428,"num_object_clones":0,"num_object_copies":-10856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5428,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004573"},"pg_stats":[{"pgid":"2.f","version":"47'15703","reported_seq":15783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.722070+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:44.722070+0000","last_peered":"2024-09-05T13:19:44.722070+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:44.722070+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21581824,"num_objects":5269,"num_object_clones":0,"num_object_copies":10538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15703,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":848}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15714","reported_seq":15824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.520897+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:48.520897+0000","last_peered":"2024-09-05T13:19:48.520897+0000","last_clean":"2024-09-05T13:19:48.520897+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:48.520897+0000","last_undegraded":"2024-09-05T13:19:48.520897+0000","last_fullsized":"2024-09-05T13:19:48.520897+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13724,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16011","reported_seq":16636,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.549789+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:48.549789+0000","last_peered":"2024-09-05T13:19:48.549789+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:48.549789+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233688,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13822,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15888","reported_seq":15963,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.533541+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:48.533541+0000","last_peered":"2024-09-05T13:19:48.533541+0000","last_clean":"2024-09-05T13:19:48.533541+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:48.533541+0000","last_undegraded":"2024-09-05T13:19:48.533541+0000","last_fullsized":"2024-09-05T13:19:48.533541+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21045248,"num_objects":5138,"num_object_clones":0,"num_object_copies":10276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15888,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15718","reported_seq":15858,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:44.647931+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:44.647931+0000","last_peered":"2024-09-05T13:19:44.647931+0000","last_clean":"2024-09-05T13:19:44.647931+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:44.647931+0000","last_undegraded":"2024-09-05T13:19:44.647931+0000","last_fullsized":"2024-09-05T13:19:44.647931+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12258,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15865","reported_seq":16007,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.779396+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:44.779396+0000","last_peered":"2024-09-05T13:19:44.779396+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:44.779396+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21667840,"num_objects":5290,"num_object_clones":0,"num_object_copies":10580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12320,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":839}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15611","reported_seq":24643,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.324603+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:48.324603+0000","last_peered":"2024-09-05T13:19:48.324603+0000","last_clean":"2024-09-05T13:19:48.324603+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:48.324603+0000","last_undegraded":"2024-09-05T13:19:48.324603+0000","last_fullsized":"2024-09-05T13:19:48.324603+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20549632,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11930,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'15879","reported_seq":16016,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:44.748172+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:44.748172+0000","last_peered":"2024-09-05T13:19:44.748172+0000","last_clean":"2024-09-05T13:19:44.748172+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:44.748172+0000","last_undegraded":"2024-09-05T13:19:44.748172+0000","last_fullsized":"2024-09-05T13:19:44.748172+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12489,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15928","reported_seq":16068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.565445+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:48.565445+0000","last_peered":"2024-09-05T13:19:48.565445+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:48.565445+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13582,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":714}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15726","reported_seq":15866,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.793960+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:44.793960+0000","last_peered":"2024-09-05T13:19:44.793960+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:44.793960+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12093,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":730}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15793","reported_seq":24699,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.541658+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:48.541658+0000","last_peered":"2024-09-05T13:19:48.541658+0000","last_clean":"2024-09-05T13:19:48.541658+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:48.541658+0000","last_undegraded":"2024-09-05T13:19:48.541658+0000","last_fullsized":"2024-09-05T13:19:48.541658+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15621","reported_seq":15749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.575263+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:48.575263+0000","last_peered":"2024-09-05T13:19:48.575263+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:48.575263+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20320256,"num_objects":4961,"num_object_clones":0,"num_object_copies":9922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13130,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":574}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15688","reported_seq":15834,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.789308+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:44.789308+0000","last_peered":"2024-09-05T13:19:44.789308+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:44.789308+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21258240,"num_objects":5190,"num_object_clones":0,"num_object_copies":10380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11839,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":771}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15847","reported_seq":16040,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.804696+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:44.804696+0000","last_peered":"2024-09-05T13:19:44.804696+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:44.804696+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21532672,"num_objects":5257,"num_object_clones":0,"num_object_copies":10514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11746,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":813}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15884","reported_seq":16028,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.809577+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:44.809577+0000","last_peered":"2024-09-05T13:19:44.809577+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:44.809577+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21282816,"num_objects":5196,"num_object_clones":0,"num_object_copies":10392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12014,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":630}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15956","reported_seq":16101,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:44.815082+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:44.815082+0000","last_peered":"2024-09-05T13:19:44.815082+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:44.815082+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21447139,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12624,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":672}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338473467,"num_objects":82637,"num_object_clones":0,"num_object_copies":165274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":208550,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9075,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":893681664,"data_stored":893601260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609732,"kb_used_data":307316,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93762108,"statfs":{"total":96636764160,"available":96012398592,"internally_reserved":0,"allocated":314691584,"data_stored":314421582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745116,"kb_used_data":321996,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93626724,"statfs":{"total":96636764160,"available":95873765376,"internally_reserved":0,"allocated":329723904,"data_stored":329453897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593316,"kb_used_data":246484,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93778524,"statfs":{"total":96636764160,"available":96029208576,"internally_reserved":0,"allocated":252399616,"data_stored":252064116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,1,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328589312,"data_stored":328585699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":251654144,"data_stored":251585038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313438208,"data_stored":313430523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:51.061 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:51.062 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15703", 'reported_seq': 15783, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:44.722070+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:44.722070+0000', 'last_peered': '2024-09-05T13:19:44.722070+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:44.722070+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21581824, 'num_objects': 5269, 'num_object_clones': 0, 'num_object_copies': 10538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5269, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15703, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 848}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:51.062 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:51.062 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16011", 'reported_seq': 16636, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:48.549789+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:19:48.549789+0000', 'last_peered': '2024-09-05T13:19:48.549789+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:48.549789+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21233688, 'num_objects': 5185, 'num_object_clones': 0, 'num_object_copies': 10370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5185, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13822, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 245, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:51.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:51.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15865", 'reported_seq': 16007, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:44.779396+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:44.779396+0000', 'last_peered': '2024-09-05T13:19:44.779396+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:44.779396+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21667840, 'num_objects': 5290, 'num_object_clones': 0, 'num_object_copies': 10580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12320, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 839}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:51.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:51.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15928", 'reported_seq': 16068, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:48.565445+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:48.565445+0000', 'last_peered': '2024-09-05T13:19:48.565445+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:48.565445+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21155840, 'num_objects': 5165, 'num_object_clones': 0, 'num_object_copies': 10330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13582, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 714}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:51.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:51.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15726", 'reported_seq': 15866, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:44.793960+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:44.793960+0000', 'last_peered': '2024-09-05T13:19:44.793960+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:44.793960+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21233664, 'num_objects': 5184, 'num_object_clones': 0, 'num_object_copies': 10368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5184, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12093, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 730}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:51.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:51.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15621", 'reported_seq': 15749, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:48.575263+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:48.575263+0000', 'last_peered': '2024-09-05T13:19:48.575263+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:48.575263+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20320256, 'num_objects': 4961, 'num_object_clones': 0, 'num_object_copies': 9922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13130, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 574}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:51.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:51.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15688", 'reported_seq': 15834, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:44.789308+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:44.789308+0000', 'last_peered': '2024-09-05T13:19:44.789308+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:44.789308+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21258240, 'num_objects': 5190, 'num_object_clones': 0, 'num_object_copies': 10380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11839, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 771}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:51.065 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:51.065 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'15847", 'reported_seq': 16040, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:44.804696+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:44.804696+0000', 'last_peered': '2024-09-05T13:19:44.804696+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:44.804696+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21532672, 'num_objects': 5257, 'num_object_clones': 0, 'num_object_copies': 10514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5257, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11746, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 813}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:51.065 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:51.065 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15884", 'reported_seq': 16028, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:44.809577+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:44.809577+0000', 'last_peered': '2024-09-05T13:19:44.809577+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:44.809577+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21282816, 'num_objects': 5196, 'num_object_clones': 0, 'num_object_copies': 10392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5196, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12014, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 630}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:51.065 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:51.065 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15956", 'reported_seq': 16101, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:44.815082+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:44.815082+0000', 'last_peered': '2024-09-05T13:19:44.815082+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:44.815082+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21447139, 'num_objects': 5237, 'num_object_clones': 0, 'num_object_copies': 10474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12624, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 672}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:51.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:51.779 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:51.779 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44423/165278 objects degraded (26.878%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":518,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":82639,"data_bytes":338473486,"bytes_used":1994919936,"bytes_avail":287915372544,"bytes_total":289910292480,"degraded_objects":44423,"degraded_total":165278,"degraded_ratio":0.26877745374459999,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":452},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:51.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:51.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:52.354 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:52.379 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-05T13:19:51.682549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":333193742,"num_objects":81350,"num_object_clones":0,"num_object_copies":162700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81350,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209841,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9076,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946772,"kb_used_data":868836,"kb_used_omap":12,"kb_used_meta":1077875,"kb_avail":281168748,"statfs":{"total":289910292480,"available":287916797952,"internally_reserved":0,"allocated":889688064,"data_stored":888812555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1103744006},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,8,4,5,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23785472,"num_objects":-5807,"num_object_clones":0,"num_object_copies":-11614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5807,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004387"},"pg_stats":[{"pgid":"2.f","version":"47'15826","reported_seq":15906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:49.816794+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:49.816794+0000","last_peered":"2024-09-05T13:19:49.816794+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:49.816794+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078016,"num_objects":5146,"num_object_clones":0,"num_object_copies":10292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15826,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":725}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15714","reported_seq":15824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.520897+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:48.520897+0000","last_peered":"2024-09-05T13:19:48.520897+0000","last_clean":"2024-09-05T13:19:48.520897+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:48.520897+0000","last_undegraded":"2024-09-05T13:19:48.520897+0000","last_fullsized":"2024-09-05T13:19:48.520897+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13724,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16011","reported_seq":16636,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.549789+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:48.549789+0000","last_peered":"2024-09-05T13:19:48.549789+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:48.549789+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233688,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13822,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15888","reported_seq":15963,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.533541+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:48.533541+0000","last_peered":"2024-09-05T13:19:48.533541+0000","last_clean":"2024-09-05T13:19:48.533541+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:48.533541+0000","last_undegraded":"2024-09-05T13:19:48.533541+0000","last_fullsized":"2024-09-05T13:19:48.533541+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21045248,"num_objects":5138,"num_object_clones":0,"num_object_copies":10276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15888,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15864","reported_seq":16004,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:49.806260+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:49.806260+0000","last_peered":"2024-09-05T13:19:49.806260+0000","last_clean":"2024-09-05T13:19:49.806260+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:49.806260+0000","last_undegraded":"2024-09-05T13:19:49.806260+0000","last_fullsized":"2024-09-05T13:19:49.806260+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20557824,"num_objects":5019,"num_object_clones":0,"num_object_copies":10038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12404,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15985","reported_seq":16129,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:49.837200+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:49.837200+0000","last_peered":"2024-09-05T13:19:49.837200+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:49.837200+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21176320,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12440,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15611","reported_seq":24643,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.324603+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:48.324603+0000","last_peered":"2024-09-05T13:19:48.324603+0000","last_clean":"2024-09-05T13:19:48.324603+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:48.324603+0000","last_undegraded":"2024-09-05T13:19:48.324603+0000","last_fullsized":"2024-09-05T13:19:48.324603+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20549632,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11930,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16027","reported_seq":16164,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:49.816162+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:49.816162+0000","last_peered":"2024-09-05T13:19:49.816162+0000","last_clean":"2024-09-05T13:19:49.816162+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:49.816162+0000","last_undegraded":"2024-09-05T13:19:49.816162+0000","last_fullsized":"2024-09-05T13:19:49.816162+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20795392,"num_objects":5077,"num_object_clones":0,"num_object_copies":10154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12637,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15928","reported_seq":16068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.565445+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:48.565445+0000","last_peered":"2024-09-05T13:19:48.565445+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:48.565445+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13582,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":714}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15870","reported_seq":16013,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:49.845909+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:49.845909+0000","last_peered":"2024-09-05T13:19:49.845909+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:49.845909+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20643840,"num_objects":5040,"num_object_clones":0,"num_object_copies":10080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12237,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":586}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15793","reported_seq":24699,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.541658+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:48.541658+0000","last_peered":"2024-09-05T13:19:48.541658+0000","last_clean":"2024-09-05T13:19:48.541658+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:48.541658+0000","last_undegraded":"2024-09-05T13:19:48.541658+0000","last_fullsized":"2024-09-05T13:19:48.541658+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15621","reported_seq":15749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:48.575263+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:48.575263+0000","last_peered":"2024-09-05T13:19:48.575263+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:48.575263+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20320256,"num_objects":4961,"num_object_clones":0,"num_object_copies":9922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13130,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":574}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15839","reported_seq":15985,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:49.817680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:49.817680+0000","last_peered":"2024-09-05T13:19:49.817680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:49.817680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20639744,"num_objects":5039,"num_object_clones":0,"num_object_copies":10078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11990,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":620}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15990","reported_seq":16183,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:49.790045+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:49.790045+0000","last_peered":"2024-09-05T13:19:49.790045+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:49.790045+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11889,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":670}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16051","reported_seq":16195,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:49.784628+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:49.784628+0000","last_peered":"2024-09-05T13:19:49.784628+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:49.784628+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20598784,"num_objects":5029,"num_object_clones":0,"num_object_copies":10058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12181,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16103","reported_seq":16247,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:49.870406+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:49.870406+0000","last_peered":"2024-09-05T13:19:49.870406+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:49.870406+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20845027,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12771,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":333193723,"num_objects":81348,"num_object_clones":0,"num_object_copies":162696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44423,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209839,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9075,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":886464512,"data_stored":886384108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609732,"kb_used_data":307316,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93762108,"statfs":{"total":96636764160,"available":96012398592,"internally_reserved":0,"allocated":314691584,"data_stored":314421582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":309653165},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743724,"kb_used_data":315036,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93628116,"statfs":{"total":96636764160,"available":95875190784,"internally_reserved":0,"allocated":322596864,"data_stored":322326857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593316,"kb_used_data":246484,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93778524,"statfs":{"total":96636764160,"available":96029208576,"internally_reserved":0,"allocated":252399616,"data_stored":252064116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,1,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":321372160,"data_stored":321368547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":251654144,"data_stored":251585038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":313438208,"data_stored":313430523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:55.381 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:55.980 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:56.005 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-05T13:19:55.684078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":323625486,"num_objects":79014,"num_object_clones":0,"num_object_copies":158028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212177,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9154,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946372,"kb_used_data":850380,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281169148,"statfs":{"total":289910292480,"available":287917207552,"internally_reserved":0,"allocated":870789120,"data_stored":869913611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1122225158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,4,5,11,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24530944,"num_objects":-5989,"num_object_clones":0,"num_object_copies":-11978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5989,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004407"},"pg_stats":[{"pgid":"2.f","version":"47'15963","reported_seq":16042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.744131+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:54.744131+0000","last_peered":"2024-09-05T13:19:54.744131+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:54.744131+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20516864,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15865","reported_seq":15975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.579125+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:53.579125+0000","last_peered":"2024-09-05T13:19:53.579125+0000","last_clean":"2024-09-05T13:19:53.579125+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:53.579125+0000","last_undegraded":"2024-09-05T13:19:53.579125+0000","last_fullsized":"2024-09-05T13:19:53.579125+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16145","reported_seq":16925,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.589788+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:53.589788+0000","last_peered":"2024-09-05T13:19:53.589788+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:53.589788+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684824,"num_objects":5051,"num_object_clones":0,"num_object_copies":10102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5051,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13956,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16064","reported_seq":16139,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.547863+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:53.547863+0000","last_peered":"2024-09-05T13:19:53.547863+0000","last_clean":"2024-09-05T13:19:53.547863+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:53.547863+0000","last_undegraded":"2024-09-05T13:19:53.547863+0000","last_fullsized":"2024-09-05T13:19:53.547863+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20324352,"num_objects":4962,"num_object_clones":0,"num_object_copies":9924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16004","reported_seq":16144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:54.781107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:54.781107+0000","last_peered":"2024-09-05T13:19:54.781107+0000","last_clean":"2024-09-05T13:19:54.781107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:54.781107+0000","last_undegraded":"2024-09-05T13:19:54.781107+0000","last_fullsized":"2024-09-05T13:19:54.781107+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19984384,"num_objects":4879,"num_object_clones":0,"num_object_copies":9758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12544,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16133","reported_seq":16277,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.872727+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:54.872727+0000","last_peered":"2024-09-05T13:19:54.872727+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:54.872727+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15777","reported_seq":24809,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.404459+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:53.404459+0000","last_peered":"2024-09-05T13:19:53.404459+0000","last_clean":"2024-09-05T13:19:53.404459+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:53.404459+0000","last_undegraded":"2024-09-05T13:19:53.404459+0000","last_fullsized":"2024-09-05T13:19:53.404459+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12096,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16170","reported_seq":16307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:54.813145+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:54.813145+0000","last_peered":"2024-09-05T13:19:54.813145+0000","last_clean":"2024-09-05T13:19:54.813145+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:54.813145+0000","last_undegraded":"2024-09-05T13:19:54.813145+0000","last_fullsized":"2024-09-05T13:19:54.813145+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20209664,"num_objects":4934,"num_object_clones":0,"num_object_copies":9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12780,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16068","reported_seq":16208,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.528916+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:53.528916+0000","last_peered":"2024-09-05T13:19:53.528916+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:53.528916+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13722,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16021","reported_seq":16163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.875600+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:54.875600+0000","last_peered":"2024-09-05T13:19:54.875600+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:54.875600+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15927","reported_seq":24833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.563712+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:53.563712+0000","last_peered":"2024-09-05T13:19:53.563712+0000","last_clean":"2024-09-05T13:19:53.563712+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:53.563712+0000","last_undegraded":"2024-09-05T13:19:53.563712+0000","last_fullsized":"2024-09-05T13:19:53.563712+0000","mapping_epoch":44,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20414464,"num_objects":4984,"num_object_clones":0,"num_object_copies":9968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15760","reported_seq":15888,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.564249+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:53.564249+0000","last_peered":"2024-09-05T13:19:53.564249+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:53.564249+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13269,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":435}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15984","reported_seq":16130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.804149+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:54.804149+0000","last_peered":"2024-09-05T13:19:54.804149+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:54.804149+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20045824,"num_objects":4894,"num_object_clones":0,"num_object_copies":9788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12135,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16141","reported_seq":16334,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.847296+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:54.847296+0000","last_peered":"2024-09-05T13:19:54.847296+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:54.847296+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20328448,"num_objects":4963,"num_object_clones":0,"num_object_copies":9926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12040,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16180","reported_seq":16324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.802024+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:54.802024+0000","last_peered":"2024-09-05T13:19:54.802024+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:54.802024+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12310,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16255","reported_seq":16400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.827220+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:54.827220+0000","last_peered":"2024-09-05T13:19:54.827220+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:54.827220+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20222435,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12923,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":323625467,"num_objects":79012,"num_object_clones":0,"num_object_copies":158024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79012,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212175,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9153,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":867381248,"data_stored":867300844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627340,"kb_used_data":302716,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744500,"statfs":{"total":96636764160,"available":95994368000,"internally_reserved":0,"allocated":309981184,"data_stored":309711182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744964,"kb_used_data":307948,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93626876,"statfs":{"total":96636764160,"available":95873921024,"internally_reserved":0,"allocated":315338752,"data_stored":315068745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574068,"kb_used_data":239716,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93797772,"statfs":{"total":96636764160,"available":96048918528,"internally_reserved":0,"allocated":245469184,"data_stored":245133684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314109952,"data_stored":314106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244539392,"data_stored":244470286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308731904,"data_stored":308724219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:56.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:56.589 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:56.591 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:56.620 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-05T13:19:55.684078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":323625486,"num_objects":79014,"num_object_clones":0,"num_object_copies":158028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212177,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9154,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946372,"kb_used_data":850380,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281169148,"statfs":{"total":289910292480,"available":287917207552,"internally_reserved":0,"allocated":870789120,"data_stored":869913611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1122225158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,4,5,11,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24530944,"num_objects":-5989,"num_object_clones":0,"num_object_copies":-11978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5989,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004407"},"pg_stats":[{"pgid":"2.f","version":"47'15963","reported_seq":16042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.744131+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:54.744131+0000","last_peered":"2024-09-05T13:19:54.744131+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:54.744131+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20516864,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15865","reported_seq":15975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.579125+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:53.579125+0000","last_peered":"2024-09-05T13:19:53.579125+0000","last_clean":"2024-09-05T13:19:53.579125+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:53.579125+0000","last_undegraded":"2024-09-05T13:19:53.579125+0000","last_fullsized":"2024-09-05T13:19:53.579125+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16145","reported_seq":16925,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.589788+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:53.589788+0000","last_peered":"2024-09-05T13:19:53.589788+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:53.589788+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684824,"num_objects":5051,"num_object_clones":0,"num_object_copies":10102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5051,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13956,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16064","reported_seq":16139,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.547863+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:53.547863+0000","last_peered":"2024-09-05T13:19:53.547863+0000","last_clean":"2024-09-05T13:19:53.547863+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:53.547863+0000","last_undegraded":"2024-09-05T13:19:53.547863+0000","last_fullsized":"2024-09-05T13:19:53.547863+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20324352,"num_objects":4962,"num_object_clones":0,"num_object_copies":9924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16004","reported_seq":16144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:54.781107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:54.781107+0000","last_peered":"2024-09-05T13:19:54.781107+0000","last_clean":"2024-09-05T13:19:54.781107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:54.781107+0000","last_undegraded":"2024-09-05T13:19:54.781107+0000","last_fullsized":"2024-09-05T13:19:54.781107+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19984384,"num_objects":4879,"num_object_clones":0,"num_object_copies":9758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12544,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16133","reported_seq":16277,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.872727+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:54.872727+0000","last_peered":"2024-09-05T13:19:54.872727+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:54.872727+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15777","reported_seq":24809,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.404459+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:53.404459+0000","last_peered":"2024-09-05T13:19:53.404459+0000","last_clean":"2024-09-05T13:19:53.404459+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:53.404459+0000","last_undegraded":"2024-09-05T13:19:53.404459+0000","last_fullsized":"2024-09-05T13:19:53.404459+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12096,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16170","reported_seq":16307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:54.813145+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:54.813145+0000","last_peered":"2024-09-05T13:19:54.813145+0000","last_clean":"2024-09-05T13:19:54.813145+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:54.813145+0000","last_undegraded":"2024-09-05T13:19:54.813145+0000","last_fullsized":"2024-09-05T13:19:54.813145+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20209664,"num_objects":4934,"num_object_clones":0,"num_object_copies":9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12780,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16068","reported_seq":16208,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.528916+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:53.528916+0000","last_peered":"2024-09-05T13:19:53.528916+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:53.528916+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13722,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16021","reported_seq":16163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.875600+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:54.875600+0000","last_peered":"2024-09-05T13:19:54.875600+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:54.875600+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15927","reported_seq":24833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.563712+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:53.563712+0000","last_peered":"2024-09-05T13:19:53.563712+0000","last_clean":"2024-09-05T13:19:53.563712+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:53.563712+0000","last_undegraded":"2024-09-05T13:19:53.563712+0000","last_fullsized":"2024-09-05T13:19:53.563712+0000","mapping_epoch":44,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20414464,"num_objects":4984,"num_object_clones":0,"num_object_copies":9968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15760","reported_seq":15888,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.564249+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:53.564249+0000","last_peered":"2024-09-05T13:19:53.564249+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:53.564249+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13269,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":435}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15984","reported_seq":16130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.804149+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:54.804149+0000","last_peered":"2024-09-05T13:19:54.804149+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:54.804149+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20045824,"num_objects":4894,"num_object_clones":0,"num_object_copies":9788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12135,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16141","reported_seq":16334,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.847296+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:54.847296+0000","last_peered":"2024-09-05T13:19:54.847296+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:54.847296+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20328448,"num_objects":4963,"num_object_clones":0,"num_object_copies":9926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12040,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16180","reported_seq":16324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.802024+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:54.802024+0000","last_peered":"2024-09-05T13:19:54.802024+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:54.802024+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12310,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16255","reported_seq":16400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.827220+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:54.827220+0000","last_peered":"2024-09-05T13:19:54.827220+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:54.827220+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20222435,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12923,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":323625467,"num_objects":79012,"num_object_clones":0,"num_object_copies":158024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79012,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212175,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9153,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":867381248,"data_stored":867300844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627340,"kb_used_data":302716,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744500,"statfs":{"total":96636764160,"available":95994368000,"internally_reserved":0,"allocated":309981184,"data_stored":309711182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744964,"kb_used_data":307948,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93626876,"statfs":{"total":96636764160,"available":95873921024,"internally_reserved":0,"allocated":315338752,"data_stored":315068745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574068,"kb_used_data":239716,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93797772,"statfs":{"total":96636764160,"available":96048918528,"internally_reserved":0,"allocated":245469184,"data_stored":245133684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314109952,"data_stored":314106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244539392,"data_stored":244470286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308731904,"data_stored":308724219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:19:56.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15963", 'reported_seq': 16042, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:54.744131+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:54.744131+0000', 'last_peered': '2024-09-05T13:19:54.744131+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:54.744131+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20516864, 'num_objects': 5009, 'num_object_clones': 0, 'num_object_copies': 10018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15963, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 588}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16145", 'reported_seq': 16925, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:53.589788+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:19:53.589788+0000', 'last_peered': '2024-09-05T13:19:53.589788+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:53.589788+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20684824, 'num_objects': 5051, 'num_object_clones': 0, 'num_object_copies': 10102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4170, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5051, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13956, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 323, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 881}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16133", 'reported_seq': 16277, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:54.872727+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:54.872727+0000', 'last_peered': '2024-09-05T13:19:54.872727+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:54.872727+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20570112, 'num_objects': 5022, 'num_object_clones': 0, 'num_object_copies': 10044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12588, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 571}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16068", 'reported_seq': 16208, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:53.528916+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:53.528916+0000', 'last_peered': '2024-09-05T13:19:53.528916+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:53.528916+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20582400, 'num_objects': 5025, 'num_object_clones': 0, 'num_object_copies': 10050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5025, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13722, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16021", 'reported_seq': 16163, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:54.875600+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:54.875600+0000', 'last_peered': '2024-09-05T13:19:54.875600+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:54.875600+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20025344, 'num_objects': 4889, 'num_object_clones': 0, 'num_object_copies': 9778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12388, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 435}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:56.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15760", 'reported_seq': 15888, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:53.564249+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:53.564249+0000', 'last_peered': '2024-09-05T13:19:53.564249+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:53.564249+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19750912, 'num_objects': 4822, 'num_object_clones': 0, 'num_object_copies': 9644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13269, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 435}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15984", 'reported_seq': 16130, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:54.804149+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:54.804149+0000', 'last_peered': '2024-09-05T13:19:54.804149+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:54.804149+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20045824, 'num_objects': 4894, 'num_object_clones': 0, 'num_object_copies': 9788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12135, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 475}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16141", 'reported_seq': 16334, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:54.847296+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:54.847296+0000', 'last_peered': '2024-09-05T13:19:54.847296+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:54.847296+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20328448, 'num_objects': 4963, 'num_object_clones': 0, 'num_object_copies': 9926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12040, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 519}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16180", 'reported_seq': 16324, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:54.802024+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:54.802024+0000', 'last_peered': '2024-09-05T13:19:54.802024+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:54.802024+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20070400, 'num_objects': 4900, 'num_object_clones': 0, 'num_object_copies': 9800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12310, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 334}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:19:56.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16255", 'reported_seq': 16400, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:54.827220+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:54.827220+0000', 'last_peered': '2024-09-05T13:19:54.827220+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:54.827220+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20222435, 'num_objects': 4938, 'num_object_clones': 0, 'num_object_copies': 9876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12923, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 373}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:19:56.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:19:57.311 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:57.311 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44345/158028 objects degraded (28.061%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":524,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":79014,"data_bytes":323625486,"bytes_used":1993084928,"bytes_avail":287917207552,"bytes_total":289910292480,"degraded_objects":44345,"degraded_total":158028,"degraded_ratio":0.28061482775204394,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":498},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:19:57.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:19:57.330 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:19:57.894 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:19:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:19:57.915 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-05T13:19:57.684903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":323625486,"num_objects":79014,"num_object_clones":0,"num_object_copies":158028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212177,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9154,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946372,"kb_used_data":850380,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281169148,"statfs":{"total":289910292480,"available":287917207552,"internally_reserved":0,"allocated":870789120,"data_stored":869913611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1122225158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,4,5,11,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19075072,"num_objects":-4657,"num_object_clones":0,"num_object_copies":-9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004437"},"pg_stats":[{"pgid":"2.f","version":"47'15963","reported_seq":16042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.744131+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:54.744131+0000","last_peered":"2024-09-05T13:19:54.744131+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:54.744131+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20516864,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15865","reported_seq":15975,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.579125+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:53.579125+0000","last_peered":"2024-09-05T13:19:53.579125+0000","last_clean":"2024-09-05T13:19:53.579125+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:53.579125+0000","last_undegraded":"2024-09-05T13:19:53.579125+0000","last_fullsized":"2024-09-05T13:19:53.579125+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16145","reported_seq":16925,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.589788+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:53.589788+0000","last_peered":"2024-09-05T13:19:53.589788+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:53.589788+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684824,"num_objects":5051,"num_object_clones":0,"num_object_copies":10102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5051,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13956,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":881}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16064","reported_seq":16139,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.547863+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:53.547863+0000","last_peered":"2024-09-05T13:19:53.547863+0000","last_clean":"2024-09-05T13:19:53.547863+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:53.547863+0000","last_undegraded":"2024-09-05T13:19:53.547863+0000","last_fullsized":"2024-09-05T13:19:53.547863+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20324352,"num_objects":4962,"num_object_clones":0,"num_object_copies":9924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16064,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16004","reported_seq":16144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:54.781107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:54.781107+0000","last_peered":"2024-09-05T13:19:54.781107+0000","last_clean":"2024-09-05T13:19:54.781107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:54.781107+0000","last_undegraded":"2024-09-05T13:19:54.781107+0000","last_fullsized":"2024-09-05T13:19:54.781107+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19984384,"num_objects":4879,"num_object_clones":0,"num_object_copies":9758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12544,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16133","reported_seq":16277,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.872727+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:54.872727+0000","last_peered":"2024-09-05T13:19:54.872727+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:54.872727+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15777","reported_seq":24809,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.404459+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:53.404459+0000","last_peered":"2024-09-05T13:19:53.404459+0000","last_clean":"2024-09-05T13:19:53.404459+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:53.404459+0000","last_undegraded":"2024-09-05T13:19:53.404459+0000","last_fullsized":"2024-09-05T13:19:53.404459+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12096,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16170","reported_seq":16307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:54.813145+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:54.813145+0000","last_peered":"2024-09-05T13:19:54.813145+0000","last_clean":"2024-09-05T13:19:54.813145+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:54.813145+0000","last_undegraded":"2024-09-05T13:19:54.813145+0000","last_fullsized":"2024-09-05T13:19:54.813145+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20209664,"num_objects":4934,"num_object_clones":0,"num_object_copies":9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12780,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16068","reported_seq":16208,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.528916+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:53.528916+0000","last_peered":"2024-09-05T13:19:53.528916+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:53.528916+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13722,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16021","reported_seq":16163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.875600+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:54.875600+0000","last_peered":"2024-09-05T13:19:54.875600+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:54.875600+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15927","reported_seq":24833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:53.563712+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:53.563712+0000","last_peered":"2024-09-05T13:19:53.563712+0000","last_clean":"2024-09-05T13:19:53.563712+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:53.563712+0000","last_undegraded":"2024-09-05T13:19:53.563712+0000","last_fullsized":"2024-09-05T13:19:53.563712+0000","mapping_epoch":44,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20414464,"num_objects":4984,"num_object_clones":0,"num_object_copies":9968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15760","reported_seq":15888,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:53.564249+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:53.564249+0000","last_peered":"2024-09-05T13:19:53.564249+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:53.564249+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13269,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":435}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15984","reported_seq":16130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.804149+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:54.804149+0000","last_peered":"2024-09-05T13:19:54.804149+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:54.804149+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20045824,"num_objects":4894,"num_object_clones":0,"num_object_copies":9788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12135,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16141","reported_seq":16334,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.847296+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:54.847296+0000","last_peered":"2024-09-05T13:19:54.847296+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:54.847296+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20328448,"num_objects":4963,"num_object_clones":0,"num_object_copies":9926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12040,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16180","reported_seq":16324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.802024+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:54.802024+0000","last_peered":"2024-09-05T13:19:54.802024+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:54.802024+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12310,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16255","reported_seq":16400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.827220+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:54.827220+0000","last_peered":"2024-09-05T13:19:54.827220+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:54.827220+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20222435,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12923,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":323625467,"num_objects":79012,"num_object_clones":0,"num_object_copies":158024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44345,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79012,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212175,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9153,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":867381248,"data_stored":867300844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627340,"kb_used_data":302716,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744500,"statfs":{"total":96636764160,"available":95994368000,"internally_reserved":0,"allocated":309981184,"data_stored":309711182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744964,"kb_used_data":307948,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93626876,"statfs":{"total":96636764160,"available":95873921024,"internally_reserved":0,"allocated":315338752,"data_stored":315068745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574068,"kb_used_data":239716,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93797772,"statfs":{"total":96636764160,"available":96048918528,"internally_reserved":0,"allocated":245469184,"data_stored":245133684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314109952,"data_stored":314106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244539392,"data_stored":244470286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":308731904,"data_stored":308724219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:00.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:01.452 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:01.453 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:01.472 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-05T13:19:59.685682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":319554062,"num_objects":78020,"num_object_clones":0,"num_object_copies":156040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213171,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9239,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1965496,"kb_used_data":838972,"kb_used_omap":12,"kb_used_meta":1126451,"kb_avail":281150024,"statfs":{"total":289910292480,"available":287897624576,"internally_reserved":0,"allocated":859107328,"data_stored":858231819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1153485830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,5,6,14,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23146496,"num_objects":-5651,"num_object_clones":0,"num_object_copies":-11302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5651,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004473"},"pg_stats":[{"pgid":"2.f","version":"47'15963","reported_seq":16042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.744131+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:54.744131+0000","last_peered":"2024-09-05T13:19:54.744131+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:54.744131+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20516864,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16033","reported_seq":16143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.641152+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:58.641152+0000","last_peered":"2024-09-05T13:19:58.641152+0000","last_clean":"2024-09-05T13:19:58.641152+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:58.641152+0000","last_undegraded":"2024-09-05T13:19:58.641152+0000","last_fullsized":"2024-09-05T13:19:58.641152+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19337216,"num_objects":4721,"num_object_clones":0,"num_object_copies":9442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14043,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16276","reported_seq":17226,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.609453+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:58.609453+0000","last_peered":"2024-09-05T13:19:58.609453+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:58.609453+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20148248,"num_objects":4920,"num_object_clones":0,"num_object_copies":9840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14087,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16218","reported_seq":16293,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.566636+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:58.566636+0000","last_peered":"2024-09-05T13:19:58.566636+0000","last_clean":"2024-09-05T13:19:58.566636+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:58.566636+0000","last_undegraded":"2024-09-05T13:19:58.566636+0000","last_fullsized":"2024-09-05T13:19:58.566636+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19693568,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16218,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16004","reported_seq":16144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:54.781107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:54.781107+0000","last_peered":"2024-09-05T13:19:54.781107+0000","last_clean":"2024-09-05T13:19:54.781107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:54.781107+0000","last_undegraded":"2024-09-05T13:19:54.781107+0000","last_fullsized":"2024-09-05T13:19:54.781107+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19984384,"num_objects":4879,"num_object_clones":0,"num_object_copies":9758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12544,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16133","reported_seq":16277,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.872727+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:54.872727+0000","last_peered":"2024-09-05T13:19:54.872727+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:54.872727+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15926","reported_seq":24958,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.430076+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:58.430076+0000","last_peered":"2024-09-05T13:19:58.430076+0000","last_clean":"2024-09-05T13:19:58.430076+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:58.430076+0000","last_undegraded":"2024-09-05T13:19:58.430076+0000","last_fullsized":"2024-09-05T13:19:58.430076+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259392,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12245,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16170","reported_seq":16307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:54.813145+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:54.813145+0000","last_peered":"2024-09-05T13:19:54.813145+0000","last_clean":"2024-09-05T13:19:54.813145+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:54.813145+0000","last_undegraded":"2024-09-05T13:19:54.813145+0000","last_fullsized":"2024-09-05T13:19:54.813145+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20209664,"num_objects":4934,"num_object_clones":0,"num_object_copies":9868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12780,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16216","reported_seq":16356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.650927+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:58.650927+0000","last_peered":"2024-09-05T13:19:58.650927+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:58.650927+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19976192,"num_objects":4877,"num_object_clones":0,"num_object_copies":9754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13870,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16021","reported_seq":16163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.875600+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:54.875600+0000","last_peered":"2024-09-05T13:19:54.875600+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:54.875600+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12388,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16061","reported_seq":24966,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.625173+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:58.625173+0000","last_peered":"2024-09-05T13:19:58.625173+0000","last_clean":"2024-09-05T13:19:58.625173+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:58.625173+0000","last_undegraded":"2024-09-05T13:19:58.625173+0000","last_fullsized":"2024-09-05T13:19:58.625173+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19865600,"num_objects":4850,"num_object_clones":0,"num_object_copies":9700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13656,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15870","reported_seq":15998,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.586220+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:58.586220+0000","last_peered":"2024-09-05T13:19:58.586220+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:58.586220+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19300352,"num_objects":4712,"num_object_clones":0,"num_object_copies":9424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13379,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15984","reported_seq":16130,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.804149+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:54.804149+0000","last_peered":"2024-09-05T13:19:54.804149+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:54.804149+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20045824,"num_objects":4894,"num_object_clones":0,"num_object_copies":9788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12135,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":475}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16141","reported_seq":16334,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.847296+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:54.847296+0000","last_peered":"2024-09-05T13:19:54.847296+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:54.847296+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20328448,"num_objects":4963,"num_object_clones":0,"num_object_copies":9926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12040,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16180","reported_seq":16324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.802024+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:54.802024+0000","last_peered":"2024-09-05T13:19:54.802024+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:54.802024+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20070400,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12310,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16255","reported_seq":16400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:54.827220+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:54.827220+0000","last_peered":"2024-09-05T13:19:54.827220+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:54.827220+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20222435,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12923,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":319554043,"num_objects":78018,"num_object_clones":0,"num_object_copies":156036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213169,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9238,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":855752704,"data_stored":855672300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626872,"kb_used_data":298084,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93744968,"statfs":{"total":96636764160,"available":95994847232,"internally_reserved":0,"allocated":305238016,"data_stored":304968014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744964,"kb_used_data":307948,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93626876,"statfs":{"total":96636764160,"available":95873921024,"internally_reserved":0,"allocated":315338752,"data_stored":315068745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593660,"kb_used_data":232940,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93778180,"statfs":{"total":96636764160,"available":96028856320,"internally_reserved":0,"allocated":238530560,"data_stored":238195060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,7,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314109952,"data_stored":314106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237608960,"data_stored":237539854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304033792,"data_stored":304026107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:01.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:02.082 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:02.105 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-05T13:20:01.686470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":314171918,"num_objects":76706,"num_object_clones":0,"num_object_copies":153412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76706,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214485,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9239,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160568,"ondisk_log_size":160568,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1945912,"kb_used_data":831876,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281169608,"statfs":{"total":289910292480,"available":287917678592,"internally_reserved":0,"allocated":851841024,"data_stored":850965515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1140706310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,8,5,8,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24301568,"num_objects":-5933,"num_object_clones":0,"num_object_copies":-11866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5933,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004505"},"pg_stats":[{"pgid":"2.f","version":"47'16134","reported_seq":16213,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.910062+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:59.910062+0000","last_peered":"2024-09-05T13:19:59.910062+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:59.910062+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16134,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16033","reported_seq":16143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.641152+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:58.641152+0000","last_peered":"2024-09-05T13:19:58.641152+0000","last_clean":"2024-09-05T13:19:58.641152+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:58.641152+0000","last_undegraded":"2024-09-05T13:19:58.641152+0000","last_fullsized":"2024-09-05T13:19:58.641152+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19337216,"num_objects":4721,"num_object_clones":0,"num_object_copies":9442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14043,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16276","reported_seq":17226,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.609453+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:58.609453+0000","last_peered":"2024-09-05T13:19:58.609453+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:58.609453+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20148248,"num_objects":4920,"num_object_clones":0,"num_object_copies":9840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14087,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16218","reported_seq":16293,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.566636+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:58.566636+0000","last_peered":"2024-09-05T13:19:58.566636+0000","last_clean":"2024-09-05T13:19:58.566636+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:58.566636+0000","last_undegraded":"2024-09-05T13:19:58.566636+0000","last_fullsized":"2024-09-05T13:19:58.566636+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19693568,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16218,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16164","reported_seq":16304,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:59.853107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:59.853107+0000","last_peered":"2024-09-05T13:19:59.853107+0000","last_clean":"2024-09-05T13:19:59.853107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:59.853107+0000","last_undegraded":"2024-09-05T13:19:59.853107+0000","last_fullsized":"2024-09-05T13:19:59.853107+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19329024,"num_objects":4719,"num_object_clones":0,"num_object_copies":9438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12704,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16301","reported_seq":16442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.841431+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:59.841431+0000","last_peered":"2024-09-05T13:19:59.841431+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:59.841431+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19881984,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12756,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15926","reported_seq":24958,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.430076+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:58.430076+0000","last_peered":"2024-09-05T13:19:58.430076+0000","last_clean":"2024-09-05T13:19:58.430076+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:58.430076+0000","last_undegraded":"2024-09-05T13:19:58.430076+0000","last_fullsized":"2024-09-05T13:19:58.430076+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259392,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12245,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16320","reported_seq":16457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:59.831815+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:59.831815+0000","last_peered":"2024-09-05T13:19:59.831815+0000","last_clean":"2024-09-05T13:19:59.831815+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:59.831815+0000","last_undegraded":"2024-09-05T13:19:59.831815+0000","last_fullsized":"2024-09-05T13:19:59.831815+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19595264,"num_objects":4784,"num_object_clones":0,"num_object_copies":9568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12930,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16216","reported_seq":16356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.650927+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:58.650927+0000","last_peered":"2024-09-05T13:19:58.650927+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:58.650927+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19976192,"num_objects":4877,"num_object_clones":0,"num_object_copies":9754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13870,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16152","reported_seq":16293,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.768799+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:59.768799+0000","last_peered":"2024-09-05T13:19:59.768799+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:59.768799+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":304}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16061","reported_seq":24966,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.625173+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:58.625173+0000","last_peered":"2024-09-05T13:19:58.625173+0000","last_clean":"2024-09-05T13:19:58.625173+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:58.625173+0000","last_undegraded":"2024-09-05T13:19:58.625173+0000","last_fullsized":"2024-09-05T13:19:58.625173+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19865600,"num_objects":4850,"num_object_clones":0,"num_object_copies":9700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13656,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15870","reported_seq":15998,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.586220+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:58.586220+0000","last_peered":"2024-09-05T13:19:58.586220+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:58.586220+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19300352,"num_objects":4712,"num_object_clones":0,"num_object_copies":9424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13379,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16104","reported_seq":16250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.860285+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:59.860285+0000","last_peered":"2024-09-05T13:19:59.860285+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:59.860285+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19554304,"num_objects":4774,"num_object_clones":0,"num_object_copies":9548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12255,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":355}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16271","reported_seq":16464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.871235+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:59.871235+0000","last_peered":"2024-09-05T13:19:59.871235+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:59.871235+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19795968,"num_objects":4833,"num_object_clones":0,"num_object_copies":9666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12170,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16316","reported_seq":16460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.844227+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:59.844227+0000","last_peered":"2024-09-05T13:19:59.844227+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:59.844227+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19513344,"num_objects":4764,"num_object_clones":0,"num_object_copies":9528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12446,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":198}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16403","reported_seq":16548,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.839714+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:59.839714+0000","last_peered":"2024-09-05T13:19:59.839714+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:59.839714+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19616227,"num_objects":4790,"num_object_clones":0,"num_object_copies":9580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13071,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":314171899,"num_objects":76704,"num_object_clones":0,"num_object_copies":153408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214483,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9238,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":848543744,"data_stored":848463340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160565,"ondisk_log_size":160565,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626872,"kb_used_data":298084,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93744968,"statfs":{"total":96636764160,"available":95994847232,"internally_reserved":0,"allocated":305238016,"data_stored":304968014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725380,"kb_used_data":300852,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93646460,"statfs":{"total":96636764160,"available":95893975040,"internally_reserved":0,"allocated":308072448,"data_stored":307802441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593660,"kb_used_data":232940,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93778180,"statfs":{"total":96636764160,"available":96028856320,"internally_reserved":0,"allocated":238530560,"data_stored":238195060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,7,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306900992,"data_stored":306897379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237608960,"data_stored":237539854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304033792,"data_stored":304026107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:02.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:02.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16134", 'reported_seq': 16213, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:59.910062+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:19:59.910062+0000', 'last_peered': '2024-09-05T13:19:59.910062+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:19:59.910062+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19816448, 'num_objects': 4838, 'num_object_clones': 0, 'num_object_copies': 9676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4838, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16134, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 417}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:02.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:02.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16276", 'reported_seq': 17226, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:58.609453+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:19:58.609453+0000', 'last_peered': '2024-09-05T13:19:58.609453+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:19:58.609453+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20148248, 'num_objects': 4920, 'num_object_clones': 0, 'num_object_copies': 9840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4085, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4920, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14087, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 408, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:02.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:02.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16301", 'reported_seq': 16442, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:59.841431+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:19:59.841431+0000', 'last_peered': '2024-09-05T13:19:59.841431+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:19:59.841431+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19881984, 'num_objects': 4854, 'num_object_clones': 0, 'num_object_copies': 9708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12756, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 403}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:02.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:02.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16216", 'reported_seq': 16356, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:58.650927+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:19:58.650927+0000', 'last_peered': '2024-09-05T13:19:58.650927+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:19:58.650927+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19976192, 'num_objects': 4877, 'num_object_clones': 0, 'num_object_copies': 9754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4877, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13870, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 426}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:02.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:02.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16152", 'reported_seq': 16293, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:59.768799+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:19:59.768799+0000', 'last_peered': '2024-09-05T13:19:59.768799+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:19:59.768799+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19488768, 'num_objects': 4758, 'num_object_clones': 0, 'num_object_copies': 9516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4758, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12519, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 304}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:02.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:02.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'15870", 'reported_seq': 15998, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:58.586220+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:19:58.586220+0000', 'last_peered': '2024-09-05T13:19:58.586220+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:19:58.586220+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19300352, 'num_objects': 4712, 'num_object_clones': 0, 'num_object_copies': 9424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4712, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13379, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 325}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:02.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:02.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16104", 'reported_seq': 16250, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:59.860285+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:19:59.860285+0000', 'last_peered': '2024-09-05T13:19:59.860285+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:19:59.860285+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19554304, 'num_objects': 4774, 'num_object_clones': 0, 'num_object_copies': 9548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4774, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12255, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 355}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:02.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:02.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16271", 'reported_seq': 16464, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:59.871235+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:19:59.871235+0000', 'last_peered': '2024-09-05T13:19:59.871235+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:19:59.871235+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19795968, 'num_objects': 4833, 'num_object_clones': 0, 'num_object_copies': 9666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12170, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 389}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:02.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:02.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16316", 'reported_seq': 16460, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:59.844227+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:19:59.844227+0000', 'last_peered': '2024-09-05T13:19:59.844227+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:19:59.844227+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19513344, 'num_objects': 4764, 'num_object_clones': 0, 'num_object_copies': 9528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12446, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 198}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:02.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:02.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16403", 'reported_seq': 16548, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:19:59.839714+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:19:59.839714+0000', 'last_peered': '2024-09-05T13:19:59.839714+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:19:59.839714+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19616227, 'num_objects': 4790, 'num_object_clones': 0, 'num_object_copies': 9580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4790, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13071, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 225}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:02.110 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:02.797 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:02.800 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44260/153412 objects degraded (28.850%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":529,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":76706,"data_bytes":314171918,"bytes_used":1992613888,"bytes_avail":287917678592,"bytes_total":289910292480,"degraded_objects":44260,"degraded_total":153412,"degraded_ratio":0.28850415873595286,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":163,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":494},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:02.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:02.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:03.381 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:03.384 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:03.416 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-05T13:20:01.686470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":314171918,"num_objects":76706,"num_object_clones":0,"num_object_copies":153412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76706,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214485,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9239,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160568,"ondisk_log_size":160568,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1945912,"kb_used_data":831876,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281169608,"statfs":{"total":289910292480,"available":287917678592,"internally_reserved":0,"allocated":851841024,"data_stored":850965515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1140706310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,8,5,8,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24301568,"num_objects":-5933,"num_object_clones":0,"num_object_copies":-11866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5933,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004505"},"pg_stats":[{"pgid":"2.f","version":"47'16134","reported_seq":16213,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.910062+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:19:59.910062+0000","last_peered":"2024-09-05T13:19:59.910062+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:19:59.910062+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19816448,"num_objects":4838,"num_object_clones":0,"num_object_copies":9676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16134,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16033","reported_seq":16143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.641152+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:19:58.641152+0000","last_peered":"2024-09-05T13:19:58.641152+0000","last_clean":"2024-09-05T13:19:58.641152+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:19:58.641152+0000","last_undegraded":"2024-09-05T13:19:58.641152+0000","last_fullsized":"2024-09-05T13:19:58.641152+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19337216,"num_objects":4721,"num_object_clones":0,"num_object_copies":9442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14043,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16276","reported_seq":17226,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.609453+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:19:58.609453+0000","last_peered":"2024-09-05T13:19:58.609453+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:19:58.609453+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20148248,"num_objects":4920,"num_object_clones":0,"num_object_copies":9840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14087,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16218","reported_seq":16293,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.566636+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:19:58.566636+0000","last_peered":"2024-09-05T13:19:58.566636+0000","last_clean":"2024-09-05T13:19:58.566636+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:19:58.566636+0000","last_undegraded":"2024-09-05T13:19:58.566636+0000","last_fullsized":"2024-09-05T13:19:58.566636+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19693568,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16218,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16164","reported_seq":16304,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:59.853107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:19:59.853107+0000","last_peered":"2024-09-05T13:19:59.853107+0000","last_clean":"2024-09-05T13:19:59.853107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:19:59.853107+0000","last_undegraded":"2024-09-05T13:19:59.853107+0000","last_fullsized":"2024-09-05T13:19:59.853107+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19329024,"num_objects":4719,"num_object_clones":0,"num_object_copies":9438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12704,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16301","reported_seq":16442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.841431+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:19:59.841431+0000","last_peered":"2024-09-05T13:19:59.841431+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:19:59.841431+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19881984,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12756,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15926","reported_seq":24958,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.430076+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:19:58.430076+0000","last_peered":"2024-09-05T13:19:58.430076+0000","last_clean":"2024-09-05T13:19:58.430076+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:19:58.430076+0000","last_undegraded":"2024-09-05T13:19:58.430076+0000","last_fullsized":"2024-09-05T13:19:58.430076+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259392,"num_objects":4702,"num_object_clones":0,"num_object_copies":9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12245,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16320","reported_seq":16457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:59.831815+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:19:59.831815+0000","last_peered":"2024-09-05T13:19:59.831815+0000","last_clean":"2024-09-05T13:19:59.831815+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:19:59.831815+0000","last_undegraded":"2024-09-05T13:19:59.831815+0000","last_fullsized":"2024-09-05T13:19:59.831815+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19595264,"num_objects":4784,"num_object_clones":0,"num_object_copies":9568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12930,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16216","reported_seq":16356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.650927+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:19:58.650927+0000","last_peered":"2024-09-05T13:19:58.650927+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:19:58.650927+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19976192,"num_objects":4877,"num_object_clones":0,"num_object_copies":9754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13870,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":426}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16152","reported_seq":16293,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.768799+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:19:59.768799+0000","last_peered":"2024-09-05T13:19:59.768799+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:19:59.768799+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":304}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16061","reported_seq":24966,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:58.625173+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:19:58.625173+0000","last_peered":"2024-09-05T13:19:58.625173+0000","last_clean":"2024-09-05T13:19:58.625173+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:19:58.625173+0000","last_undegraded":"2024-09-05T13:19:58.625173+0000","last_fullsized":"2024-09-05T13:19:58.625173+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19865600,"num_objects":4850,"num_object_clones":0,"num_object_copies":9700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13656,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15870","reported_seq":15998,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:58.586220+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:19:58.586220+0000","last_peered":"2024-09-05T13:19:58.586220+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:19:58.586220+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19300352,"num_objects":4712,"num_object_clones":0,"num_object_copies":9424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13379,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":325}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16104","reported_seq":16250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.860285+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:19:59.860285+0000","last_peered":"2024-09-05T13:19:59.860285+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:19:59.860285+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19554304,"num_objects":4774,"num_object_clones":0,"num_object_copies":9548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12255,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":355}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16271","reported_seq":16464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.871235+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:19:59.871235+0000","last_peered":"2024-09-05T13:19:59.871235+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:19:59.871235+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19795968,"num_objects":4833,"num_object_clones":0,"num_object_copies":9666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12170,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16316","reported_seq":16460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.844227+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:19:59.844227+0000","last_peered":"2024-09-05T13:19:59.844227+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:19:59.844227+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19513344,"num_objects":4764,"num_object_clones":0,"num_object_copies":9528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12446,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":198}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16403","reported_seq":16548,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:19:59.839714+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:19:59.839714+0000","last_peered":"2024-09-05T13:19:59.839714+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:19:59.839714+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19616227,"num_objects":4790,"num_object_clones":0,"num_object_copies":9580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13071,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":314171899,"num_objects":76704,"num_object_clones":0,"num_object_copies":153408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214483,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9238,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":848543744,"data_stored":848463340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160565,"ondisk_log_size":160565,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626872,"kb_used_data":298084,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93744968,"statfs":{"total":96636764160,"available":95994847232,"internally_reserved":0,"allocated":305238016,"data_stored":304968014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725380,"kb_used_data":300852,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93646460,"statfs":{"total":96636764160,"available":95893975040,"internally_reserved":0,"allocated":308072448,"data_stored":307802441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593660,"kb_used_data":232940,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93778180,"statfs":{"total":96636764160,"available":96028856320,"internally_reserved":0,"allocated":238530560,"data_stored":238195060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,7,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306900992,"data_stored":306897379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237608960,"data_stored":237539854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304033792,"data_stored":304026107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:06.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:06.993 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:06.993 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:07.016 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-05T13:20:05.687709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":304804366,"num_objects":74419,"num_object_clones":0,"num_object_copies":148838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74419,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216772,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9311,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1927604,"kb_used_data":813564,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281187916,"statfs":{"total":289910292480,"available":287936425984,"internally_reserved":0,"allocated":833089536,"data_stored":832214027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1140706310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,3,5,7,14,7,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24129536,"num_objects":-5891,"num_object_clones":0,"num_object_copies":-11782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5891,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004533"},"pg_stats":[{"pgid":"2.f","version":"47'16283","reported_seq":16361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.867678+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:04.867678+0000","last_peered":"2024-09-05T13:20:04.867678+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:04.867678+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19206144,"num_objects":4689,"num_object_clones":0,"num_object_copies":9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16283,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16184","reported_seq":16294,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.640938+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:03.640938+0000","last_peered":"2024-09-05T13:20:03.640938+0000","last_clean":"2024-09-05T13:20:03.640938+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:03.640938+0000","last_undegraded":"2024-09-05T13:20:03.640938+0000","last_fullsized":"2024-09-05T13:20:03.640938+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14194,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16434","reported_seq":17525,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.655189+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:03.655189+0000","last_peered":"2024-09-05T13:20:03.655189+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:03.655189+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501080,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14245,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":749}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16351","reported_seq":16426,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.579089+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:03.579089+0000","last_peered":"2024-09-05T13:20:03.579089+0000","last_clean":"2024-09-05T13:20:03.579089+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:03.579089+0000","last_undegraded":"2024-09-05T13:20:03.579089+0000","last_fullsized":"2024-09-05T13:20:03.579089+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19148800,"num_objects":4675,"num_object_clones":0,"num_object_copies":9350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16351,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16321","reported_seq":16461,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:04.951991+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:04.951991+0000","last_peered":"2024-09-05T13:20:04.951991+0000","last_clean":"2024-09-05T13:20:04.951991+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:04.951991+0000","last_undegraded":"2024-09-05T13:20:04.951991+0000","last_fullsized":"2024-09-05T13:20:04.951991+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18685952,"num_objects":4562,"num_object_clones":0,"num_object_copies":9124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12861,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16422","reported_seq":16566,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.887573+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:04.887573+0000","last_peered":"2024-09-05T13:20:04.887573+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:04.887573+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19386368,"num_objects":4733,"num_object_clones":0,"num_object_copies":9466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12877,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16059","reported_seq":25091,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.561620+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:03.561620+0000","last_peered":"2024-09-05T13:20:03.561620+0000","last_clean":"2024-09-05T13:20:03.561620+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:03.561620+0000","last_undegraded":"2024-09-05T13:20:03.561620+0000","last_fullsized":"2024-09-05T13:20:03.561620+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12378,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16470","reported_seq":16607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:04.901901+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:04.901901+0000","last_peered":"2024-09-05T13:20:04.901901+0000","last_clean":"2024-09-05T13:20:04.901901+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:04.901901+0000","last_undegraded":"2024-09-05T13:20:04.901901+0000","last_fullsized":"2024-09-05T13:20:04.901901+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18980864,"num_objects":4634,"num_object_clones":0,"num_object_copies":9268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16358","reported_seq":16498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.568269+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:03.568269+0000","last_peered":"2024-09-05T13:20:03.568269+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:03.568269+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19394560,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14012,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16291","reported_seq":16433,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.896469+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:04.896469+0000","last_peered":"2024-09-05T13:20:04.896469+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:04.896469+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18919424,"num_objects":4619,"num_object_clones":0,"num_object_copies":9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16207","reported_seq":25113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.583093+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:03.583093+0000","last_peered":"2024-09-05T13:20:03.583093+0000","last_clean":"2024-09-05T13:20:03.583093+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:03.583093+0000","last_undegraded":"2024-09-05T13:20:03.583093+0000","last_fullsized":"2024-09-05T13:20:03.583093+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13802,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16013","reported_seq":16141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.613458+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:03.613458+0000","last_peered":"2024-09-05T13:20:03.613458+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:03.613458+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16249","reported_seq":16393,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.948204+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:04.948204+0000","last_peered":"2024-09-05T13:20:04.948204+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:04.948204+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12400,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16424","reported_seq":16617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.912444+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:04.912444+0000","last_peered":"2024-09-05T13:20:04.912444+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:04.912444+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19169280,"num_objects":4680,"num_object_clones":0,"num_object_copies":9360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12323,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16444","reported_seq":16588,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.949830+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:04.949830+0000","last_peered":"2024-09-05T13:20:04.949830+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:04.949830+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12574,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16542","reported_seq":16687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.701775+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:04.701775+0000","last_peered":"2024-09-05T13:20:04.701775+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:04.701775+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046883,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13210,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":86}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":304804347,"num_objects":74417,"num_object_clones":0,"num_object_copies":148834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216770,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9310,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":829677568,"data_stored":829597164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609720,"kb_used_data":293416,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93762120,"statfs":{"total":96636764160,"available":96012410880,"internally_reserved":0,"allocated":300457984,"data_stored":300187982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323874477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722752,"kb_used_data":294064,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93649088,"statfs":{"total":96636764160,"available":95896666112,"internally_reserved":0,"allocated":301121536,"data_stored":300851529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,3,7,2,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595132,"kb_used_data":226084,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93776708,"statfs":{"total":96636764160,"available":96027348992,"internally_reserved":0,"allocated":231510016,"data_stored":231174516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":377876143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299913216,"data_stored":299909603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230580224,"data_stored":230511118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299184128,"data_stored":299176443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:07.017 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:07.594 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:07.594 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:07.622 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-05T13:20:05.687709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":304804366,"num_objects":74419,"num_object_clones":0,"num_object_copies":148838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74419,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216772,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9311,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1927604,"kb_used_data":813564,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281187916,"statfs":{"total":289910292480,"available":287936425984,"internally_reserved":0,"allocated":833089536,"data_stored":832214027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1140706310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,3,5,7,14,7,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24129536,"num_objects":-5891,"num_object_clones":0,"num_object_copies":-11782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5891,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004533"},"pg_stats":[{"pgid":"2.f","version":"47'16283","reported_seq":16361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.867678+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:04.867678+0000","last_peered":"2024-09-05T13:20:04.867678+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:04.867678+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19206144,"num_objects":4689,"num_object_clones":0,"num_object_copies":9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16283,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16184","reported_seq":16294,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.640938+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:03.640938+0000","last_peered":"2024-09-05T13:20:03.640938+0000","last_clean":"2024-09-05T13:20:03.640938+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:03.640938+0000","last_undegraded":"2024-09-05T13:20:03.640938+0000","last_fullsized":"2024-09-05T13:20:03.640938+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14194,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16434","reported_seq":17525,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.655189+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:03.655189+0000","last_peered":"2024-09-05T13:20:03.655189+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:03.655189+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501080,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14245,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":749}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16351","reported_seq":16426,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.579089+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:03.579089+0000","last_peered":"2024-09-05T13:20:03.579089+0000","last_clean":"2024-09-05T13:20:03.579089+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:03.579089+0000","last_undegraded":"2024-09-05T13:20:03.579089+0000","last_fullsized":"2024-09-05T13:20:03.579089+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19148800,"num_objects":4675,"num_object_clones":0,"num_object_copies":9350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16351,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16321","reported_seq":16461,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:04.951991+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:04.951991+0000","last_peered":"2024-09-05T13:20:04.951991+0000","last_clean":"2024-09-05T13:20:04.951991+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:04.951991+0000","last_undegraded":"2024-09-05T13:20:04.951991+0000","last_fullsized":"2024-09-05T13:20:04.951991+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18685952,"num_objects":4562,"num_object_clones":0,"num_object_copies":9124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12861,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16422","reported_seq":16566,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.887573+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:04.887573+0000","last_peered":"2024-09-05T13:20:04.887573+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:04.887573+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19386368,"num_objects":4733,"num_object_clones":0,"num_object_copies":9466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12877,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16059","reported_seq":25091,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.561620+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:03.561620+0000","last_peered":"2024-09-05T13:20:03.561620+0000","last_clean":"2024-09-05T13:20:03.561620+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:03.561620+0000","last_undegraded":"2024-09-05T13:20:03.561620+0000","last_fullsized":"2024-09-05T13:20:03.561620+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12378,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16470","reported_seq":16607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:04.901901+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:04.901901+0000","last_peered":"2024-09-05T13:20:04.901901+0000","last_clean":"2024-09-05T13:20:04.901901+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:04.901901+0000","last_undegraded":"2024-09-05T13:20:04.901901+0000","last_fullsized":"2024-09-05T13:20:04.901901+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18980864,"num_objects":4634,"num_object_clones":0,"num_object_copies":9268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16358","reported_seq":16498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.568269+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:03.568269+0000","last_peered":"2024-09-05T13:20:03.568269+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:03.568269+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19394560,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14012,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16291","reported_seq":16433,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.896469+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:04.896469+0000","last_peered":"2024-09-05T13:20:04.896469+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:04.896469+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18919424,"num_objects":4619,"num_object_clones":0,"num_object_copies":9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16207","reported_seq":25113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.583093+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:03.583093+0000","last_peered":"2024-09-05T13:20:03.583093+0000","last_clean":"2024-09-05T13:20:03.583093+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:03.583093+0000","last_undegraded":"2024-09-05T13:20:03.583093+0000","last_fullsized":"2024-09-05T13:20:03.583093+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13802,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16013","reported_seq":16141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.613458+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:03.613458+0000","last_peered":"2024-09-05T13:20:03.613458+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:03.613458+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16249","reported_seq":16393,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.948204+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:04.948204+0000","last_peered":"2024-09-05T13:20:04.948204+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:04.948204+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12400,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16424","reported_seq":16617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.912444+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:04.912444+0000","last_peered":"2024-09-05T13:20:04.912444+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:04.912444+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19169280,"num_objects":4680,"num_object_clones":0,"num_object_copies":9360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12323,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16444","reported_seq":16588,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.949830+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:04.949830+0000","last_peered":"2024-09-05T13:20:04.949830+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:04.949830+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12574,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16542","reported_seq":16687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.701775+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:04.701775+0000","last_peered":"2024-09-05T13:20:04.701775+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:04.701775+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046883,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13210,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":86}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":304804347,"num_objects":74417,"num_object_clones":0,"num_object_copies":148834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216770,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9310,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":829677568,"data_stored":829597164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609720,"kb_used_data":293416,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93762120,"statfs":{"total":96636764160,"available":96012410880,"internally_reserved":0,"allocated":300457984,"data_stored":300187982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323874477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722752,"kb_used_data":294064,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93649088,"statfs":{"total":96636764160,"available":95896666112,"internally_reserved":0,"allocated":301121536,"data_stored":300851529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,3,7,2,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595132,"kb_used_data":226084,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93776708,"statfs":{"total":96636764160,"available":96027348992,"internally_reserved":0,"allocated":231510016,"data_stored":231174516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":377876143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299913216,"data_stored":299909603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230580224,"data_stored":230511118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299184128,"data_stored":299176443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:07.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:07.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16283", 'reported_seq': 16361, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:04.867678+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:04.867678+0000', 'last_peered': '2024-09-05T13:20:04.867678+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:04.867678+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19206144, 'num_objects': 4689, 'num_object_clones': 0, 'num_object_copies': 9378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4689, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16283, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 268}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:07.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:07.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16434", 'reported_seq': 17525, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:03.655189+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:03.655189+0000', 'last_peered': '2024-09-05T13:20:03.655189+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:03.655189+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19501080, 'num_objects': 4762, 'num_object_clones': 0, 'num_object_copies': 9524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4013, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4762, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14245, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 480, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 749}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:07.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:07.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16422", 'reported_seq': 16566, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:04.887573+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:04.887573+0000', 'last_peered': '2024-09-05T13:20:04.887573+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:04.887573+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19386368, 'num_objects': 4733, 'num_object_clones': 0, 'num_object_copies': 9466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12877, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 282}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:07.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:07.624 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16358", 'reported_seq': 16498, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:03.568269+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:03.568269+0000', 'last_peered': '2024-09-05T13:20:03.568269+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:03.568269+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19394560, 'num_objects': 4735, 'num_object_clones': 0, 'num_object_copies': 9470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4735, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14012, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16291", 'reported_seq': 16433, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:04.896469+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:04.896469+0000', 'last_peered': '2024-09-05T13:20:04.896469+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:04.896469+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18919424, 'num_objects': 4619, 'num_object_clones': 0, 'num_object_copies': 9238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12658, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 165}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'16013", 'reported_seq': 16141, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:03.613458+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:03.613458+0000', 'last_peered': '2024-09-05T13:20:03.613458+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:03.613458+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18714624, 'num_objects': 4569, 'num_object_clones': 0, 'num_object_copies': 9138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4569, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13522, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 182}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16249", 'reported_seq': 16393, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:04.948204+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:04.948204+0000', 'last_peered': '2024-09-05T13:20:04.948204+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:04.948204+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18960384, 'num_objects': 4629, 'num_object_clones': 0, 'num_object_copies': 9258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12400, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 210}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16424", 'reported_seq': 16617, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:04.912444+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:04.912444+0000', 'last_peered': '2024-09-05T13:20:04.912444+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:04.912444+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19169280, 'num_objects': 4680, 'num_object_clones': 0, 'num_object_copies': 9360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4680, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12323, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 236}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16444", 'reported_seq': 16588, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:04.949830+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:04.949830+0000', 'last_peered': '2024-09-05T13:20:04.949830+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:04.949830+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18989056, 'num_objects': 4636, 'num_object_clones': 0, 'num_object_copies': 9272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4636, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12574, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 70}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:07.625 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:07.626 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16542", 'reported_seq': 16687, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:04.701775+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:04.701775+0000', 'last_peered': '2024-09-05T13:20:04.701775+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:04.701775+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19046883, 'num_objects': 4651, 'num_object_clones': 0, 'num_object_copies': 9302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13210, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 86}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:07.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:08.313 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:08.313 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44188/148838 objects degraded (29.689%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":535,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":74419,"data_bytes":304804366,"bytes_used":1973866496,"bytes_avail":287936425984,"bytes_total":289910292480,"degraded_objects":44188,"degraded_total":148838,"degraded_ratio":0.29688654779021484,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":382},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:08.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:08.337 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:08.920 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:08.920 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:08.943 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-05T13:20:07.688536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":304804366,"num_objects":74419,"num_object_clones":0,"num_object_copies":148838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74419,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216772,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9311,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1927604,"kb_used_data":813564,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281187916,"statfs":{"total":289910292480,"available":287936425984,"internally_reserved":0,"allocated":833089536,"data_stored":832214027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1140706310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,3,5,7,14,7,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18821120,"num_objects":-4595,"num_object_clones":0,"num_object_copies":-9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4595,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004457"},"pg_stats":[{"pgid":"2.f","version":"47'16283","reported_seq":16361,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.867678+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:04.867678+0000","last_peered":"2024-09-05T13:20:04.867678+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:04.867678+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19206144,"num_objects":4689,"num_object_clones":0,"num_object_copies":9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16283,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16184","reported_seq":16294,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.640938+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:03.640938+0000","last_peered":"2024-09-05T13:20:03.640938+0000","last_clean":"2024-09-05T13:20:03.640938+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:03.640938+0000","last_undegraded":"2024-09-05T13:20:03.640938+0000","last_fullsized":"2024-09-05T13:20:03.640938+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14194,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16434","reported_seq":17525,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.655189+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:03.655189+0000","last_peered":"2024-09-05T13:20:03.655189+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:03.655189+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501080,"num_objects":4762,"num_object_clones":0,"num_object_copies":9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4762,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14245,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":749}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16351","reported_seq":16426,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.579089+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:03.579089+0000","last_peered":"2024-09-05T13:20:03.579089+0000","last_clean":"2024-09-05T13:20:03.579089+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:03.579089+0000","last_undegraded":"2024-09-05T13:20:03.579089+0000","last_fullsized":"2024-09-05T13:20:03.579089+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19148800,"num_objects":4675,"num_object_clones":0,"num_object_copies":9350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16351,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16321","reported_seq":16461,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:04.951991+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:04.951991+0000","last_peered":"2024-09-05T13:20:04.951991+0000","last_clean":"2024-09-05T13:20:04.951991+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:04.951991+0000","last_undegraded":"2024-09-05T13:20:04.951991+0000","last_fullsized":"2024-09-05T13:20:04.951991+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18685952,"num_objects":4562,"num_object_clones":0,"num_object_copies":9124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12861,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16422","reported_seq":16566,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.887573+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:04.887573+0000","last_peered":"2024-09-05T13:20:04.887573+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:04.887573+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19386368,"num_objects":4733,"num_object_clones":0,"num_object_copies":9466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12877,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16059","reported_seq":25091,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.561620+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:03.561620+0000","last_peered":"2024-09-05T13:20:03.561620+0000","last_clean":"2024-09-05T13:20:03.561620+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:03.561620+0000","last_undegraded":"2024-09-05T13:20:03.561620+0000","last_fullsized":"2024-09-05T13:20:03.561620+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12378,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16470","reported_seq":16607,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:04.901901+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:04.901901+0000","last_peered":"2024-09-05T13:20:04.901901+0000","last_clean":"2024-09-05T13:20:04.901901+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:04.901901+0000","last_undegraded":"2024-09-05T13:20:04.901901+0000","last_fullsized":"2024-09-05T13:20:04.901901+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18980864,"num_objects":4634,"num_object_clones":0,"num_object_copies":9268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13080,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16358","reported_seq":16498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.568269+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:03.568269+0000","last_peered":"2024-09-05T13:20:03.568269+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:03.568269+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19394560,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14012,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":284}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16291","reported_seq":16433,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.896469+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:04.896469+0000","last_peered":"2024-09-05T13:20:04.896469+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:04.896469+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18919424,"num_objects":4619,"num_object_clones":0,"num_object_copies":9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":165}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16207","reported_seq":25113,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:03.583093+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:03.583093+0000","last_peered":"2024-09-05T13:20:03.583093+0000","last_clean":"2024-09-05T13:20:03.583093+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:03.583093+0000","last_undegraded":"2024-09-05T13:20:03.583093+0000","last_fullsized":"2024-09-05T13:20:03.583093+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13802,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16013","reported_seq":16141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:03.613458+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:03.613458+0000","last_peered":"2024-09-05T13:20:03.613458+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:03.613458+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13522,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16249","reported_seq":16393,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.948204+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:04.948204+0000","last_peered":"2024-09-05T13:20:04.948204+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:04.948204+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12400,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16424","reported_seq":16617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.912444+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:04.912444+0000","last_peered":"2024-09-05T13:20:04.912444+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:04.912444+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19169280,"num_objects":4680,"num_object_clones":0,"num_object_copies":9360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12323,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16444","reported_seq":16588,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.949830+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:04.949830+0000","last_peered":"2024-09-05T13:20:04.949830+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:04.949830+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12574,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":70}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16542","reported_seq":16687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:04.701775+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:04.701775+0000","last_peered":"2024-09-05T13:20:04.701775+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:04.701775+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046883,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13210,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":86}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":304804347,"num_objects":74417,"num_object_clones":0,"num_object_copies":148834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216770,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9310,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":829677568,"data_stored":829597164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609720,"kb_used_data":293416,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93762120,"statfs":{"total":96636764160,"available":96012410880,"internally_reserved":0,"allocated":300457984,"data_stored":300187982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323874477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":722752,"kb_used_data":294064,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93649088,"statfs":{"total":96636764160,"available":95896666112,"internally_reserved":0,"allocated":301121536,"data_stored":300851529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,3,7,2,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595132,"kb_used_data":226084,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93776708,"statfs":{"total":96636764160,"available":96027348992,"internally_reserved":0,"allocated":231510016,"data_stored":231174516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":377876143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299913216,"data_stored":299909603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230580224,"data_stored":230511118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299184128,"data_stored":299176443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:11.946 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:12.536 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:12.538 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:12.558 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-05T13:20:11.690114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":295252494,"num_objects":72087,"num_object_clones":0,"num_object_copies":144174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219104,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9388,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1912460,"kb_used_data":794240,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281203060,"statfs":{"total":289910292480,"available":287951933440,"internally_reserved":0,"allocated":813301760,"data_stored":812426251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1144966150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,6,10,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24301568,"num_objects":-5933,"num_object_clones":0,"num_object_copies":-11866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5933,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004432"},"pg_stats":[{"pgid":"2.f","version":"47'16426","reported_seq":16506,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.998119+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:09.998119+0000","last_peered":"2024-09-05T13:20:09.998119+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:09.998119+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18620416,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16426,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":125}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16328","reported_seq":16438,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.609012+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:08.609012+0000","last_peered":"2024-09-05T13:20:08.609012+0000","last_clean":"2024-09-05T13:20:08.609012+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:08.609012+0000","last_undegraded":"2024-09-05T13:20:08.609012+0000","last_fullsized":"2024-09-05T13:20:08.609012+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18128896,"num_objects":4426,"num_object_clones":0,"num_object_copies":8852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14338,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16579","reported_seq":17827,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.682736+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:08.682736+0000","last_peered":"2024-09-05T13:20:08.682736+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:08.682736+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18907160,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14390,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":681}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16501","reported_seq":16576,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.666897+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:08.666897+0000","last_peered":"2024-09-05T13:20:08.666897+0000","last_clean":"2024-09-05T13:20:08.666897+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:08.666897+0000","last_undegraded":"2024-09-05T13:20:08.666897+0000","last_fullsized":"2024-09-05T13:20:08.666897+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16501,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16463","reported_seq":16603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:09.982085+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:09.982085+0000","last_peered":"2024-09-05T13:20:09.982085+0000","last_clean":"2024-09-05T13:20:09.982085+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:09.982085+0000","last_undegraded":"2024-09-05T13:20:09.982085+0000","last_fullsized":"2024-09-05T13:20:09.982085+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18104320,"num_objects":4420,"num_object_clones":0,"num_object_copies":8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16574","reported_seq":16716,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.012209+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:10.012209+0000","last_peered":"2024-09-05T13:20:10.012209+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:10.012209+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13029,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16224","reported_seq":25256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.692442+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:08.692442+0000","last_peered":"2024-09-05T13:20:08.692442+0000","last_clean":"2024-09-05T13:20:08.692442+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:08.692442+0000","last_undegraded":"2024-09-05T13:20:08.692442+0000","last_fullsized":"2024-09-05T13:20:08.692442+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18038784,"num_objects":4404,"num_object_clones":0,"num_object_copies":8808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12543,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16627","reported_seq":16764,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:09.952915+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:09.952915+0000","last_peered":"2024-09-05T13:20:09.952915+0000","last_clean":"2024-09-05T13:20:09.952915+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:09.952915+0000","last_undegraded":"2024-09-05T13:20:09.952915+0000","last_fullsized":"2024-09-05T13:20:09.952915+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18337792,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13237,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16503","reported_seq":16643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.631286+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:08.631286+0000","last_peered":"2024-09-05T13:20:08.631286+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:08.631286+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14157,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16446","reported_seq":16589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.926077+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:09.926077+0000","last_peered":"2024-09-05T13:20:09.926077+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:09.926077+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12813,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16351","reported_seq":25257,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.601370+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:08.601370+0000","last_peered":"2024-09-05T13:20:08.601370+0000","last_clean":"2024-09-05T13:20:08.601370+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:08.601370+0000","last_undegraded":"2024-09-05T13:20:08.601370+0000","last_fullsized":"2024-09-05T13:20:08.601370+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13946,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16158","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.589045+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:08.589045+0000","last_peered":"2024-09-05T13:20:08.589045+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:08.589045+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18120704,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13667,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16386","reported_seq":16532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.952677+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:09.952677+0000","last_peered":"2024-09-05T13:20:09.952677+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:09.952677+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18399232,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12537,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":73}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16553","reported_seq":16746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.002549+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:10.002549+0000","last_peered":"2024-09-05T13:20:10.002549+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:10.002549+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12452,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16599","reported_seq":16743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.030920+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:10.030920+0000","last_peered":"2024-09-05T13:20:10.030920+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:10.030920+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18354176,"num_objects":4481,"num_object_clones":0,"num_object_copies":8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12729,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-85}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16666","reported_seq":16811,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.951160+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:09.951160+0000","last_peered":"2024-09-05T13:20:09.951160+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:09.951160+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18538979,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":295252475,"num_objects":72085,"num_object_clones":0,"num_object_copies":144170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72085,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219102,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9387,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":810532864,"data_stored":810452460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612548,"kb_used_data":287924,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93759292,"statfs":{"total":96636764160,"available":96009515008,"internally_reserved":0,"allocated":294834176,"data_stored":294564174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":724068,"kb_used_data":287036,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93647772,"statfs":{"total":96636764160,"available":95895318528,"internally_reserved":0,"allocated":293924864,"data_stored":293654857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575844,"kb_used_data":219280,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93795996,"statfs":{"total":96636764160,"available":96047099904,"internally_reserved":0,"allocated":224542720,"data_stored":224207220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292786176,"data_stored":292782563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223309824,"data_stored":223240718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294436864,"data_stored":294429179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:12.559 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:13.154 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:13.156 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:13.178 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-05T13:20:11.690114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":295252494,"num_objects":72087,"num_object_clones":0,"num_object_copies":144174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219104,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9388,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1912460,"kb_used_data":794240,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281203060,"statfs":{"total":289910292480,"available":287951933440,"internally_reserved":0,"allocated":813301760,"data_stored":812426251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1144966150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,6,10,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24301568,"num_objects":-5933,"num_object_clones":0,"num_object_copies":-11866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5933,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004432"},"pg_stats":[{"pgid":"2.f","version":"47'16426","reported_seq":16506,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.998119+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:09.998119+0000","last_peered":"2024-09-05T13:20:09.998119+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:09.998119+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18620416,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16426,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":125}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16328","reported_seq":16438,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.609012+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:08.609012+0000","last_peered":"2024-09-05T13:20:08.609012+0000","last_clean":"2024-09-05T13:20:08.609012+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:08.609012+0000","last_undegraded":"2024-09-05T13:20:08.609012+0000","last_fullsized":"2024-09-05T13:20:08.609012+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18128896,"num_objects":4426,"num_object_clones":0,"num_object_copies":8852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14338,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16579","reported_seq":17827,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.682736+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:08.682736+0000","last_peered":"2024-09-05T13:20:08.682736+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:08.682736+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18907160,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14390,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":681}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16501","reported_seq":16576,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.666897+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:08.666897+0000","last_peered":"2024-09-05T13:20:08.666897+0000","last_clean":"2024-09-05T13:20:08.666897+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:08.666897+0000","last_undegraded":"2024-09-05T13:20:08.666897+0000","last_fullsized":"2024-09-05T13:20:08.666897+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16501,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16463","reported_seq":16603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:09.982085+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:09.982085+0000","last_peered":"2024-09-05T13:20:09.982085+0000","last_clean":"2024-09-05T13:20:09.982085+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:09.982085+0000","last_undegraded":"2024-09-05T13:20:09.982085+0000","last_fullsized":"2024-09-05T13:20:09.982085+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18104320,"num_objects":4420,"num_object_clones":0,"num_object_copies":8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16574","reported_seq":16716,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.012209+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:10.012209+0000","last_peered":"2024-09-05T13:20:10.012209+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:10.012209+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13029,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16224","reported_seq":25256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.692442+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:08.692442+0000","last_peered":"2024-09-05T13:20:08.692442+0000","last_clean":"2024-09-05T13:20:08.692442+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:08.692442+0000","last_undegraded":"2024-09-05T13:20:08.692442+0000","last_fullsized":"2024-09-05T13:20:08.692442+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18038784,"num_objects":4404,"num_object_clones":0,"num_object_copies":8808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12543,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16627","reported_seq":16764,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:09.952915+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:09.952915+0000","last_peered":"2024-09-05T13:20:09.952915+0000","last_clean":"2024-09-05T13:20:09.952915+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:09.952915+0000","last_undegraded":"2024-09-05T13:20:09.952915+0000","last_fullsized":"2024-09-05T13:20:09.952915+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18337792,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13237,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16503","reported_seq":16643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.631286+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:08.631286+0000","last_peered":"2024-09-05T13:20:08.631286+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:08.631286+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14157,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16446","reported_seq":16589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.926077+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:09.926077+0000","last_peered":"2024-09-05T13:20:09.926077+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:09.926077+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12813,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16351","reported_seq":25257,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.601370+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:08.601370+0000","last_peered":"2024-09-05T13:20:08.601370+0000","last_clean":"2024-09-05T13:20:08.601370+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:08.601370+0000","last_undegraded":"2024-09-05T13:20:08.601370+0000","last_fullsized":"2024-09-05T13:20:08.601370+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13946,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16158","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.589045+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:08.589045+0000","last_peered":"2024-09-05T13:20:08.589045+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:08.589045+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18120704,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13667,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16386","reported_seq":16532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.952677+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:09.952677+0000","last_peered":"2024-09-05T13:20:09.952677+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:09.952677+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18399232,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12537,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":73}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16553","reported_seq":16746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.002549+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:10.002549+0000","last_peered":"2024-09-05T13:20:10.002549+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:10.002549+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12452,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16599","reported_seq":16743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.030920+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:10.030920+0000","last_peered":"2024-09-05T13:20:10.030920+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:10.030920+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18354176,"num_objects":4481,"num_object_clones":0,"num_object_copies":8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12729,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-85}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16666","reported_seq":16811,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.951160+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:09.951160+0000","last_peered":"2024-09-05T13:20:09.951160+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:09.951160+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18538979,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":295252475,"num_objects":72085,"num_object_clones":0,"num_object_copies":144170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72085,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219102,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9387,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":810532864,"data_stored":810452460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612548,"kb_used_data":287924,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93759292,"statfs":{"total":96636764160,"available":96009515008,"internally_reserved":0,"allocated":294834176,"data_stored":294564174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":724068,"kb_used_data":287036,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93647772,"statfs":{"total":96636764160,"available":95895318528,"internally_reserved":0,"allocated":293924864,"data_stored":293654857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575844,"kb_used_data":219280,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93795996,"statfs":{"total":96636764160,"available":96047099904,"internally_reserved":0,"allocated":224542720,"data_stored":224207220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292786176,"data_stored":292782563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223309824,"data_stored":223240718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294436864,"data_stored":294429179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:13.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:13.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16426", 'reported_seq': 16506, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:09.998119+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:09.998119+0000', 'last_peered': '2024-09-05T13:20:09.998119+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:09.998119+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18620416, 'num_objects': 4546, 'num_object_clones': 0, 'num_object_copies': 9092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16426, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 125}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:13.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:13.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16579", 'reported_seq': 17827, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:08.682736+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:08.682736+0000', 'last_peered': '2024-09-05T13:20:08.682736+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:08.682736+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18907160, 'num_objects': 4617, 'num_object_clones': 0, 'num_object_copies': 9234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3936, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4617, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14390, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 557, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 681}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:13.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:13.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16574", 'reported_seq': 16716, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:10.012209+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:10.012209+0000', 'last_peered': '2024-09-05T13:20:10.012209+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:10.012209+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18763776, 'num_objects': 4581, 'num_object_clones': 0, 'num_object_copies': 9162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13029, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:13.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:13.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16503", 'reported_seq': 16643, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:08.631286+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:08.631286+0000', 'last_peered': '2024-09-05T13:20:08.631286+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:08.631286+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18800640, 'num_objects': 4590, 'num_object_clones': 0, 'num_object_copies': 9180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14157, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16446", 'reported_seq': 16589, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:09.926077+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:09.926077+0000', 'last_peered': '2024-09-05T13:20:09.926077+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:09.926077+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18284544, 'num_objects': 4464, 'num_object_clones': 0, 'num_object_copies': 8928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12813, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 10}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'16158", 'reported_seq': 16286, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:08.589045+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:08.589045+0000', 'last_peered': '2024-09-05T13:20:08.589045+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:08.589045+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18120704, 'num_objects': 4424, 'num_object_clones': 0, 'num_object_copies': 8848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4424, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13667, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 37}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16386", 'reported_seq': 16532, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:09.952677+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:09.952677+0000', 'last_peered': '2024-09-05T13:20:09.952677+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:09.952677+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18399232, 'num_objects': 4492, 'num_object_clones': 0, 'num_object_copies': 8984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12537, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 73}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16553", 'reported_seq': 16746, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:10.002549+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:10.002549+0000', 'last_peered': '2024-09-05T13:20:10.002549+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:10.002549+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18640896, 'num_objects': 4551, 'num_object_clones': 0, 'num_object_copies': 9102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4551, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12452, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:13.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16599", 'reported_seq': 16743, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:10.030920+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:10.030920+0000', 'last_peered': '2024-09-05T13:20:10.030920+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:10.030920+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18354176, 'num_objects': 4481, 'num_object_clones': 0, 'num_object_copies': 8962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4481, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12729, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -85}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:13.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:13.182 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16666", 'reported_seq': 16811, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:09.951160+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:09.951160+0000', 'last_peered': '2024-09-05T13:20:09.951160+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:09.951160+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18538979, 'num_objects': 4527, 'num_object_clones': 0, 'num_object_copies': 9054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13334, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -38}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:13.182 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:13.904 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:13.904 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44111/144174 objects degraded (30.596%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":540,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72087,"data_bytes":295252494,"bytes_used":1958359040,"bytes_avail":287951933440,"bytes_total":289910292480,"degraded_objects":44111,"degraded_total":144174,"degraded_ratio":0.30595669122033098,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":494},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:13.933 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:13.934 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:14.483 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:14.506 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-05T13:20:13.690919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":295252494,"num_objects":72087,"num_object_clones":0,"num_object_copies":144174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219104,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9388,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1912460,"kb_used_data":794240,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281203060,"statfs":{"total":289910292480,"available":287951933440,"internally_reserved":0,"allocated":813301760,"data_stored":812426251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1144966150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,6,10,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18919424,"num_objects":-4619,"num_object_clones":0,"num_object_copies":-9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4619,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004449"},"pg_stats":[{"pgid":"2.f","version":"47'16426","reported_seq":16506,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.998119+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:09.998119+0000","last_peered":"2024-09-05T13:20:09.998119+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:09.998119+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18620416,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16426,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":125}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16328","reported_seq":16438,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.609012+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:08.609012+0000","last_peered":"2024-09-05T13:20:08.609012+0000","last_clean":"2024-09-05T13:20:08.609012+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:08.609012+0000","last_undegraded":"2024-09-05T13:20:08.609012+0000","last_fullsized":"2024-09-05T13:20:08.609012+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18128896,"num_objects":4426,"num_object_clones":0,"num_object_copies":8852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14338,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16579","reported_seq":17827,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.682736+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:08.682736+0000","last_peered":"2024-09-05T13:20:08.682736+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:08.682736+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18907160,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14390,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":681}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16501","reported_seq":16576,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.666897+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:08.666897+0000","last_peered":"2024-09-05T13:20:08.666897+0000","last_clean":"2024-09-05T13:20:08.666897+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:08.666897+0000","last_undegraded":"2024-09-05T13:20:08.666897+0000","last_fullsized":"2024-09-05T13:20:08.666897+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16501,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16463","reported_seq":16603,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:09.982085+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:09.982085+0000","last_peered":"2024-09-05T13:20:09.982085+0000","last_clean":"2024-09-05T13:20:09.982085+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:09.982085+0000","last_undegraded":"2024-09-05T13:20:09.982085+0000","last_fullsized":"2024-09-05T13:20:09.982085+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18104320,"num_objects":4420,"num_object_clones":0,"num_object_copies":8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13003,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16574","reported_seq":16716,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.012209+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:10.012209+0000","last_peered":"2024-09-05T13:20:10.012209+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:10.012209+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13029,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16224","reported_seq":25256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.692442+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:08.692442+0000","last_peered":"2024-09-05T13:20:08.692442+0000","last_clean":"2024-09-05T13:20:08.692442+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:08.692442+0000","last_undegraded":"2024-09-05T13:20:08.692442+0000","last_fullsized":"2024-09-05T13:20:08.692442+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18038784,"num_objects":4404,"num_object_clones":0,"num_object_copies":8808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12543,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16627","reported_seq":16764,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:09.952915+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:09.952915+0000","last_peered":"2024-09-05T13:20:09.952915+0000","last_clean":"2024-09-05T13:20:09.952915+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:09.952915+0000","last_undegraded":"2024-09-05T13:20:09.952915+0000","last_fullsized":"2024-09-05T13:20:09.952915+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18337792,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13237,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16503","reported_seq":16643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.631286+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:08.631286+0000","last_peered":"2024-09-05T13:20:08.631286+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:08.631286+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18800640,"num_objects":4590,"num_object_clones":0,"num_object_copies":9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14157,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16446","reported_seq":16589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.926077+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:09.926077+0000","last_peered":"2024-09-05T13:20:09.926077+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:09.926077+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12813,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":10}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16351","reported_seq":25257,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:08.601370+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:08.601370+0000","last_peered":"2024-09-05T13:20:08.601370+0000","last_clean":"2024-09-05T13:20:08.601370+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:08.601370+0000","last_undegraded":"2024-09-05T13:20:08.601370+0000","last_fullsized":"2024-09-05T13:20:08.601370+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18677760,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13946,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16158","reported_seq":16286,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:08.589045+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:08.589045+0000","last_peered":"2024-09-05T13:20:08.589045+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:08.589045+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18120704,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13667,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16386","reported_seq":16532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.952677+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:09.952677+0000","last_peered":"2024-09-05T13:20:09.952677+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:09.952677+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18399232,"num_objects":4492,"num_object_clones":0,"num_object_copies":8984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12537,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":73}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16553","reported_seq":16746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.002549+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:10.002549+0000","last_peered":"2024-09-05T13:20:10.002549+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:10.002549+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18640896,"num_objects":4551,"num_object_clones":0,"num_object_copies":9102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12452,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16599","reported_seq":16743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:10.030920+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:10.030920+0000","last_peered":"2024-09-05T13:20:10.030920+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:10.030920+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18354176,"num_objects":4481,"num_object_clones":0,"num_object_copies":8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12729,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-85}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16666","reported_seq":16811,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:09.951160+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:09.951160+0000","last_peered":"2024-09-05T13:20:09.951160+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:09.951160+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18538979,"num_objects":4527,"num_object_clones":0,"num_object_copies":9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13334,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-38}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":295252475,"num_objects":72085,"num_object_clones":0,"num_object_copies":144170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72085,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219102,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9387,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":810532864,"data_stored":810452460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612548,"kb_used_data":287924,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93759292,"statfs":{"total":96636764160,"available":96009515008,"internally_reserved":0,"allocated":294834176,"data_stored":294564174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":724068,"kb_used_data":287036,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93647772,"statfs":{"total":96636764160,"available":95895318528,"internally_reserved":0,"allocated":293924864,"data_stored":293654857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575844,"kb_used_data":219280,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93795996,"statfs":{"total":96636764160,"available":96047099904,"internally_reserved":0,"allocated":224542720,"data_stored":224207220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292786176,"data_stored":292782563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223309824,"data_stored":223240718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294436864,"data_stored":294429179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:17.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:18.059 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:18.060 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:18.082 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-05T13:20:17.692547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":285717006,"num_objects":69759,"num_object_clones":0,"num_object_copies":139518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":221432,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9464,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1893756,"kb_used_data":775552,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281221764,"statfs":{"total":289910292480,"available":287971086336,"internally_reserved":0,"allocated":794165248,"data_stored":793289739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1144966150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,12,8,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19087360,"num_objects":-4660,"num_object_clones":0,"num_object_copies":-9320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4660,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004837"},"pg_stats":[{"pgid":"2.f","version":"47'16571","reported_seq":16651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.015099+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:15.015099+0000","last_peered":"2024-09-05T13:20:15.015099+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:15.015099+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18026496,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16571,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16471","reported_seq":16581,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:13.560332+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:13.560332+0000","last_peered":"2024-09-05T13:20:13.560332+0000","last_clean":"2024-09-05T13:20:13.560332+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:13.560332+0000","last_undegraded":"2024-09-05T13:20:13.560332+0000","last_fullsized":"2024-09-05T13:20:13.560332+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543168,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14481,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16715","reported_seq":18116,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:13.671140+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:13.671140+0000","last_peered":"2024-09-05T13:20:13.671140+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:13.671140+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18350104,"num_objects":4481,"num_object_clones":0,"num_object_copies":8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14526,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":621}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16660","reported_seq":16735,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:13.581091+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:13.581091+0000","last_peered":"2024-09-05T13:20:13.581091+0000","last_clean":"2024-09-05T13:20:13.581091+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:13.581091+0000","last_undegraded":"2024-09-05T13:20:13.581091+0000","last_fullsized":"2024-09-05T13:20:13.581091+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16660,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16617","reported_seq":16757,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:14.975497+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:14.975497+0000","last_peered":"2024-09-05T13:20:14.975497+0000","last_clean":"2024-09-05T13:20:14.975497+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:14.975497+0000","last_undegraded":"2024-09-05T13:20:14.975497+0000","last_fullsized":"2024-09-05T13:20:14.975497+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17473536,"num_objects":4266,"num_object_clones":0,"num_object_copies":8532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13157,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16738","reported_seq":16882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.030222+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:15.030222+0000","last_peered":"2024-09-05T13:20:15.030222+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:15.030222+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18092032,"num_objects":4417,"num_object_clones":0,"num_object_copies":8834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13193,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16374","reported_seq":25406,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:13.871043+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:13.871043+0000","last_peered":"2024-09-05T13:20:13.871043+0000","last_clean":"2024-09-05T13:20:13.871043+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:13.871043+0000","last_undegraded":"2024-09-05T13:20:13.871043+0000","last_fullsized":"2024-09-05T13:20:13.871043+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17424384,"num_objects":4254,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12693,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16779","reported_seq":16916,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:14.906854+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:14.906854+0000","last_peered":"2024-09-05T13:20:14.906854+0000","last_clean":"2024-09-05T13:20:14.906854+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:14.906854+0000","last_undegraded":"2024-09-05T13:20:14.906854+0000","last_fullsized":"2024-09-05T13:20:14.906854+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17715200,"num_objects":4325,"num_object_clones":0,"num_object_copies":8650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13389,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16657","reported_seq":16797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:13.651870+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:13.651870+0000","last_peered":"2024-09-05T13:20:13.651870+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:13.651870+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18169856,"num_objects":4436,"num_object_clones":0,"num_object_copies":8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14311,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-15}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16577","reported_seq":16720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.034183+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:15.034183+0000","last_peered":"2024-09-05T13:20:15.034183+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:15.034183+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17747968,"num_objects":4333,"num_object_clones":0,"num_object_copies":8666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-121}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16479","reported_seq":25385,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:13.549000+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:13.549000+0000","last_peered":"2024-09-05T13:20:13.549000+0000","last_clean":"2024-09-05T13:20:13.549000+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:13.549000+0000","last_undegraded":"2024-09-05T13:20:13.549000+0000","last_fullsized":"2024-09-05T13:20:13.549000+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":4432,"num_object_clones":0,"num_object_copies":8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16294","reported_seq":16422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:13.652070+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:13.652070+0000","last_peered":"2024-09-05T13:20:13.652070+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:13.652070+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17563648,"num_objects":4288,"num_object_clones":0,"num_object_copies":8576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13803,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-99}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16539","reported_seq":16685,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.051766+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:15.051766+0000","last_peered":"2024-09-05T13:20:15.051766+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:15.051766+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17772544,"num_objects":4339,"num_object_clones":0,"num_object_copies":8678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12690,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-80}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16682","reported_seq":16875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.015211+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:15.015211+0000","last_peered":"2024-09-05T13:20:15.015211+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:15.015211+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112512,"num_objects":4422,"num_object_clones":0,"num_object_copies":8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12581,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-22}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16737","reported_seq":16879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:14.904141+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:14.904141+0000","last_peered":"2024-09-05T13:20:14.904141+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:14.904141+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12867,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-223}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16822","reported_seq":16967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:14.930464+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:14.930464+0000","last_peered":"2024-09-05T13:20:14.930464+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:14.930464+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900003,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":285716987,"num_objects":69757,"num_object_clones":0,"num_object_copies":139514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":221430,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9463,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791244800,"data_stored":791164396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160912,"ondisk_log_size":160912,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611984,"kb_used_data":283200,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93759856,"statfs":{"total":96636764160,"available":96010092544,"internally_reserved":0,"allocated":289996800,"data_stored":289726798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704540,"kb_used_data":280004,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93667300,"statfs":{"total":96636764160,"available":95915315200,"internally_reserved":0,"allocated":286724096,"data_stored":286454089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577232,"kb_used_data":212348,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93794608,"statfs":{"total":96636764160,"available":96045678592,"internally_reserved":0,"allocated":217444352,"data_stored":217108852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285556736,"data_stored":285553123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":216096768,"data_stored":216027662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":289591296,"data_stored":289583611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:18.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:18.650 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:18.651 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:18.678 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-05T13:20:17.692547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":285717006,"num_objects":69759,"num_object_clones":0,"num_object_copies":139518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":221432,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9464,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1893756,"kb_used_data":775552,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281221764,"statfs":{"total":289910292480,"available":287971086336,"internally_reserved":0,"allocated":794165248,"data_stored":793289739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1144966150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,12,8,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19087360,"num_objects":-4660,"num_object_clones":0,"num_object_copies":-9320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4660,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004837"},"pg_stats":[{"pgid":"2.f","version":"47'16571","reported_seq":16651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.015099+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:15.015099+0000","last_peered":"2024-09-05T13:20:15.015099+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:15.015099+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18026496,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16571,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16471","reported_seq":16581,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:13.560332+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:13.560332+0000","last_peered":"2024-09-05T13:20:13.560332+0000","last_clean":"2024-09-05T13:20:13.560332+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:13.560332+0000","last_undegraded":"2024-09-05T13:20:13.560332+0000","last_fullsized":"2024-09-05T13:20:13.560332+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543168,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14481,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16715","reported_seq":18116,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:13.671140+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:13.671140+0000","last_peered":"2024-09-05T13:20:13.671140+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:13.671140+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18350104,"num_objects":4481,"num_object_clones":0,"num_object_copies":8962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14526,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":633,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":621}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16660","reported_seq":16735,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:13.581091+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:13.581091+0000","last_peered":"2024-09-05T13:20:13.581091+0000","last_clean":"2024-09-05T13:20:13.581091+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:13.581091+0000","last_undegraded":"2024-09-05T13:20:13.581091+0000","last_fullsized":"2024-09-05T13:20:13.581091+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16660,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16617","reported_seq":16757,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:14.975497+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:14.975497+0000","last_peered":"2024-09-05T13:20:14.975497+0000","last_clean":"2024-09-05T13:20:14.975497+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:14.975497+0000","last_undegraded":"2024-09-05T13:20:14.975497+0000","last_fullsized":"2024-09-05T13:20:14.975497+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17473536,"num_objects":4266,"num_object_clones":0,"num_object_copies":8532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13157,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16738","reported_seq":16882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.030222+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:15.030222+0000","last_peered":"2024-09-05T13:20:15.030222+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:15.030222+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18092032,"num_objects":4417,"num_object_clones":0,"num_object_copies":8834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13193,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16374","reported_seq":25406,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:13.871043+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:13.871043+0000","last_peered":"2024-09-05T13:20:13.871043+0000","last_clean":"2024-09-05T13:20:13.871043+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:13.871043+0000","last_undegraded":"2024-09-05T13:20:13.871043+0000","last_fullsized":"2024-09-05T13:20:13.871043+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17424384,"num_objects":4254,"num_object_clones":0,"num_object_copies":8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12693,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16779","reported_seq":16916,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:14.906854+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:14.906854+0000","last_peered":"2024-09-05T13:20:14.906854+0000","last_clean":"2024-09-05T13:20:14.906854+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:14.906854+0000","last_undegraded":"2024-09-05T13:20:14.906854+0000","last_fullsized":"2024-09-05T13:20:14.906854+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17715200,"num_objects":4325,"num_object_clones":0,"num_object_copies":8650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13389,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16657","reported_seq":16797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:13.651870+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:13.651870+0000","last_peered":"2024-09-05T13:20:13.651870+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:13.651870+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18169856,"num_objects":4436,"num_object_clones":0,"num_object_copies":8872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14311,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-15}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16577","reported_seq":16720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.034183+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:15.034183+0000","last_peered":"2024-09-05T13:20:15.034183+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:15.034183+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17747968,"num_objects":4333,"num_object_clones":0,"num_object_copies":8666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-121}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16479","reported_seq":25385,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:13.549000+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:13.549000+0000","last_peered":"2024-09-05T13:20:13.549000+0000","last_clean":"2024-09-05T13:20:13.549000+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:13.549000+0000","last_undegraded":"2024-09-05T13:20:13.549000+0000","last_fullsized":"2024-09-05T13:20:13.549000+0000","mapping_epoch":44,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18153472,"num_objects":4432,"num_object_clones":0,"num_object_copies":8864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16294","reported_seq":16422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:13.652070+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:13.652070+0000","last_peered":"2024-09-05T13:20:13.652070+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:13.652070+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17563648,"num_objects":4288,"num_object_clones":0,"num_object_copies":8576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13803,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-99}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16539","reported_seq":16685,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.051766+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:15.051766+0000","last_peered":"2024-09-05T13:20:15.051766+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:15.051766+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17772544,"num_objects":4339,"num_object_clones":0,"num_object_copies":8678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12690,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-80}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16682","reported_seq":16875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.015211+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:15.015211+0000","last_peered":"2024-09-05T13:20:15.015211+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:15.015211+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112512,"num_objects":4422,"num_object_clones":0,"num_object_copies":8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12581,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-22}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16737","reported_seq":16879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:14.904141+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:14.904141+0000","last_peered":"2024-09-05T13:20:14.904141+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:14.904141+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12867,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-223}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16822","reported_seq":16967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:14.930464+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:14.930464+0000","last_peered":"2024-09-05T13:20:14.930464+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:14.930464+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900003,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":285716987,"num_objects":69757,"num_object_clones":0,"num_object_copies":139514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":221430,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9463,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791244800,"data_stored":791164396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160912,"ondisk_log_size":160912,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611984,"kb_used_data":283200,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93759856,"statfs":{"total":96636764160,"available":96010092544,"internally_reserved":0,"allocated":289996800,"data_stored":289726798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704540,"kb_used_data":280004,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93667300,"statfs":{"total":96636764160,"available":95915315200,"internally_reserved":0,"allocated":286724096,"data_stored":286454089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577232,"kb_used_data":212348,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93794608,"statfs":{"total":96636764160,"available":96045678592,"internally_reserved":0,"allocated":217444352,"data_stored":217108852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285556736,"data_stored":285553123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":216096768,"data_stored":216027662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":289591296,"data_stored":289583611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:18.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:18.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16571", 'reported_seq': 16651, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:15.015099+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:15.015099+0000', 'last_peered': '2024-09-05T13:20:15.015099+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:15.015099+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18026496, 'num_objects': 4401, 'num_object_clones': 0, 'num_object_copies': 8802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16571, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -20}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:18.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:18.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16715", 'reported_seq': 18116, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:13.671140+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:13.671140+0000', 'last_peered': '2024-09-05T13:20:13.671140+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:13.671140+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18350104, 'num_objects': 4481, 'num_object_clones': 0, 'num_object_copies': 8962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3860, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4481, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14526, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 633, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 621}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:18.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:18.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16738", 'reported_seq': 16882, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:15.030222+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:15.030222+0000', 'last_peered': '2024-09-05T13:20:15.030222+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:15.030222+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18092032, 'num_objects': 4417, 'num_object_clones': 0, 'num_object_copies': 8834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4417, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13193, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -34}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:18.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:18.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16657", 'reported_seq': 16797, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:13.651870+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:13.651870+0000', 'last_peered': '2024-09-05T13:20:13.651870+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:13.651870+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18169856, 'num_objects': 4436, 'num_object_clones': 0, 'num_object_copies': 8872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14311, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -15}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:18.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:18.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16577", 'reported_seq': 16720, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:15.034183+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:15.034183+0000', 'last_peered': '2024-09-05T13:20:15.034183+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:15.034183+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17747968, 'num_objects': 4333, 'num_object_clones': 0, 'num_object_copies': 8666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12944, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -121}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:18.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:18.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'16294", 'reported_seq': 16422, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:13.652070+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:13.652070+0000', 'last_peered': '2024-09-05T13:20:13.652070+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:13.652070+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17563648, 'num_objects': 4288, 'num_object_clones': 0, 'num_object_copies': 8576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4288, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13803, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -99}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:18.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:18.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16539", 'reported_seq': 16685, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:15.051766+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:15.051766+0000', 'last_peered': '2024-09-05T13:20:15.051766+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:15.051766+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17772544, 'num_objects': 4339, 'num_object_clones': 0, 'num_object_copies': 8678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12690, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -80}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:18.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:18.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16682", 'reported_seq': 16875, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:15.015211+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:15.015211+0000', 'last_peered': '2024-09-05T13:20:15.015211+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:15.015211+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18112512, 'num_objects': 4422, 'num_object_clones': 0, 'num_object_copies': 8844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4422, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12581, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -22}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:18.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:18.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16737", 'reported_seq': 16879, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:14.904141+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:14.904141+0000', 'last_peered': '2024-09-05T13:20:14.904141+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:14.904141+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17788928, 'num_objects': 4343, 'num_object_clones': 0, 'num_object_copies': 8686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12867, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -223}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:18.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:18.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16822", 'reported_seq': 16967, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:14.930464+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:14.930464+0000', 'last_peered': '2024-09-05T13:20:14.930464+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:14.930464+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17900003, 'num_objects': 4371, 'num_object_clones': 0, 'num_object_copies': 8742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13490, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -194}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:18.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:19.390 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:19.390 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 44035/139518 objects degraded (31.562%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":546,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":69759,"data_bytes":285717006,"bytes_used":1939206144,"bytes_avail":287971086336,"bytes_total":289910292480,"degraded_objects":44035,"degraded_total":139518,"degraded_ratio":0.31562235697186025,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":388},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:19.417 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:19.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:19.972 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:19.974 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:19.994 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-05T13:20:19.693306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":281592334,"num_objects":68752,"num_object_clones":0,"num_object_copies":137504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222439,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9546,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1870568,"kb_used_data":764844,"kb_used_omap":12,"kb_used_meta":1105651,"kb_avail":281244952,"statfs":{"total":289910292480,"available":287994830848,"internally_reserved":0,"allocated":783200256,"data_stored":782324747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1132186630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,6,6,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23212032,"num_objects":-5667,"num_object_clones":0,"num_object_copies":-11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004769"},"pg_stats":[{"pgid":"2.f","version":"47'16571","reported_seq":16651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.015099+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:15.015099+0000","last_peered":"2024-09-05T13:20:15.015099+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:15.015099+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18026496,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16571,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16605","reported_seq":16715,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.601320+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:18.601320+0000","last_peered":"2024-09-05T13:20:18.601320+0000","last_clean":"2024-09-05T13:20:18.601320+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:18.601320+0000","last_undegraded":"2024-09-05T13:20:18.601320+0000","last_fullsized":"2024-09-05T13:20:18.601320+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14615,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16873","reported_seq":18438,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.685211+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:18.685211+0000","last_peered":"2024-09-05T13:20:18.685211+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:18.685211+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17702936,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3778,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14684,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16819","reported_seq":16894,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.667010+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:18.667010+0000","last_peered":"2024-09-05T13:20:18.667010+0000","last_clean":"2024-09-05T13:20:18.667010+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:18.667010+0000","last_undegraded":"2024-09-05T13:20:18.667010+0000","last_fullsized":"2024-09-05T13:20:18.667010+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16819,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16617","reported_seq":16757,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:14.975497+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:14.975497+0000","last_peered":"2024-09-05T13:20:14.975497+0000","last_clean":"2024-09-05T13:20:14.975497+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:14.975497+0000","last_undegraded":"2024-09-05T13:20:14.975497+0000","last_fullsized":"2024-09-05T13:20:14.975497+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17473536,"num_objects":4266,"num_object_clones":0,"num_object_copies":8532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13157,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16738","reported_seq":16882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.030222+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:15.030222+0000","last_peered":"2024-09-05T13:20:15.030222+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:15.030222+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18092032,"num_objects":4417,"num_object_clones":0,"num_object_copies":8834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13193,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16513","reported_seq":25544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.884533+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:18.884533+0000","last_peered":"2024-09-05T13:20:18.884533+0000","last_clean":"2024-09-05T13:20:18.884533+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:18.884533+0000","last_undegraded":"2024-09-05T13:20:18.884533+0000","last_fullsized":"2024-09-05T13:20:18.884533+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16855040,"num_objects":4115,"num_object_clones":0,"num_object_copies":8230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12832,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16779","reported_seq":16916,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:14.906854+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:14.906854+0000","last_peered":"2024-09-05T13:20:14.906854+0000","last_clean":"2024-09-05T13:20:14.906854+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:14.906854+0000","last_undegraded":"2024-09-05T13:20:14.906854+0000","last_fullsized":"2024-09-05T13:20:14.906854+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17715200,"num_objects":4325,"num_object_clones":0,"num_object_copies":8650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13389,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16795","reported_seq":16935,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.604761+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:18.604761+0000","last_peered":"2024-09-05T13:20:18.604761+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:18.604761+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14449,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-153}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16577","reported_seq":16720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.034183+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:15.034183+0000","last_peered":"2024-09-05T13:20:15.034183+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:15.034183+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17747968,"num_objects":4333,"num_object_clones":0,"num_object_copies":8666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-121}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16620","reported_seq":25526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.438525+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:18.438525+0000","last_peered":"2024-09-05T13:20:18.438525+0000","last_clean":"2024-09-05T13:20:18.438525+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:18.438525+0000","last_undegraded":"2024-09-05T13:20:18.438525+0000","last_fullsized":"2024-09-05T13:20:18.438525+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16432","reported_seq":16560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.608437+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:18.608437+0000","last_peered":"2024-09-05T13:20:18.608437+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:18.608437+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16998400,"num_objects":4150,"num_object_clones":0,"num_object_copies":8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13941,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-237}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16539","reported_seq":16685,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.051766+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:15.051766+0000","last_peered":"2024-09-05T13:20:15.051766+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:15.051766+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17772544,"num_objects":4339,"num_object_clones":0,"num_object_copies":8678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12690,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-80}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16682","reported_seq":16875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:15.015211+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:15.015211+0000","last_peered":"2024-09-05T13:20:15.015211+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:15.015211+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18112512,"num_objects":4422,"num_object_clones":0,"num_object_copies":8844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12581,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-22}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16737","reported_seq":16879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:14.904141+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:14.904141+0000","last_peered":"2024-09-05T13:20:14.904141+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:14.904141+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12867,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-223}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16822","reported_seq":16967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:14.930464+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:14.930464+0000","last_peered":"2024-09-05T13:20:14.930464+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:14.930464+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900003,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":281592315,"num_objects":68750,"num_object_clones":0,"num_object_copies":137500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222437,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9545,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":779223040,"data_stored":779142636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160719,"ondisk_log_size":160719,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608112,"kb_used_data":279328,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93763728,"statfs":{"total":96636764160,"available":96014057472,"internally_reserved":0,"allocated":286031872,"data_stored":285761870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704540,"kb_used_data":280004,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93667300,"statfs":{"total":96636764160,"available":95915315200,"internally_reserved":0,"allocated":286724096,"data_stored":286454089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557916,"kb_used_data":205512,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93813924,"statfs":{"total":96636764160,"available":96065458176,"internally_reserved":0,"allocated":210444288,"data_stored":210108788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285556736,"data_stored":285553123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208842752,"data_stored":208773646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284823552,"data_stored":284815867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:22.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:23.544 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:23.547 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:23.570 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-05T13:20:21.694137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276263438,"num_objects":67451,"num_object_clones":0,"num_object_copies":134902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223740,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9546,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160723,"ondisk_log_size":160723,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871924,"kb_used_data":757888,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281243596,"statfs":{"total":289910292480,"available":287993442304,"internally_reserved":0,"allocated":776077312,"data_stored":775201803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1140706310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,6,3,11],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24289280,"num_objects":-5930,"num_object_clones":0,"num_object_copies":-11860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5930,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004840"},"pg_stats":[{"pgid":"2.f","version":"47'16711","reported_seq":16791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.055412+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:20.055412+0000","last_peered":"2024-09-05T13:20:20.055412+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:20.055412+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17453056,"num_objects":4261,"num_object_clones":0,"num_object_copies":8522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16711,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-160}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16605","reported_seq":16715,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.601320+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:18.601320+0000","last_peered":"2024-09-05T13:20:18.601320+0000","last_clean":"2024-09-05T13:20:18.601320+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:18.601320+0000","last_undegraded":"2024-09-05T13:20:18.601320+0000","last_fullsized":"2024-09-05T13:20:18.601320+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14615,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16873","reported_seq":18438,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.685211+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:18.685211+0000","last_peered":"2024-09-05T13:20:18.685211+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:18.685211+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17702936,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3778,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14684,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16819","reported_seq":16894,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.667010+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:18.667010+0000","last_peered":"2024-09-05T13:20:18.667010+0000","last_clean":"2024-09-05T13:20:18.667010+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:18.667010+0000","last_undegraded":"2024-09-05T13:20:18.667010+0000","last_fullsized":"2024-09-05T13:20:18.667010+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16819,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16756","reported_seq":16896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:20.033862+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:20.033862+0000","last_peered":"2024-09-05T13:20:20.033862+0000","last_clean":"2024-09-05T13:20:20.033862+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:20.033862+0000","last_undegraded":"2024-09-05T13:20:20.033862+0000","last_fullsized":"2024-09-05T13:20:20.033862+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16904192,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13296,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16884","reported_seq":17028,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.026383+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:20.026383+0000","last_peered":"2024-09-05T13:20:20.026383+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:20.026383+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13339,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16513","reported_seq":25544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.884533+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:18.884533+0000","last_peered":"2024-09-05T13:20:18.884533+0000","last_clean":"2024-09-05T13:20:18.884533+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:18.884533+0000","last_undegraded":"2024-09-05T13:20:18.884533+0000","last_fullsized":"2024-09-05T13:20:18.884533+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16855040,"num_objects":4115,"num_object_clones":0,"num_object_copies":8230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12832,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16916","reported_seq":17053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:19.884802+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:19.884802+0000","last_peered":"2024-09-05T13:20:19.884802+0000","last_clean":"2024-09-05T13:20:19.884802+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:19.884802+0000","last_undegraded":"2024-09-05T13:20:19.884802+0000","last_fullsized":"2024-09-05T13:20:19.884802+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17154048,"num_objects":4188,"num_object_clones":0,"num_object_copies":8376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16795","reported_seq":16935,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.604761+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:18.604761+0000","last_peered":"2024-09-05T13:20:18.604761+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:18.604761+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14449,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-153}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16714","reported_seq":16857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.058064+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:20.058064+0000","last_peered":"2024-09-05T13:20:20.058064+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:20.058064+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17186816,"num_objects":4196,"num_object_clones":0,"num_object_copies":8392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13081,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16620","reported_seq":25526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.438525+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:18.438525+0000","last_peered":"2024-09-05T13:20:18.438525+0000","last_clean":"2024-09-05T13:20:18.438525+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:18.438525+0000","last_undegraded":"2024-09-05T13:20:18.438525+0000","last_fullsized":"2024-09-05T13:20:18.438525+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16432","reported_seq":16560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.608437+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:18.608437+0000","last_peered":"2024-09-05T13:20:18.608437+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:18.608437+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16998400,"num_objects":4150,"num_object_clones":0,"num_object_copies":8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13941,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-237}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16686","reported_seq":16832,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:19.947213+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:19.947213+0000","last_peered":"2024-09-05T13:20:19.947213+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:19.947213+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12837,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-227}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16821","reported_seq":17014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.056438+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:20.056438+0000","last_peered":"2024-09-05T13:20:20.056438+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:20.056438+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543168,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12720,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16900","reported_seq":17037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.012888+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:20.012888+0000","last_peered":"2024-09-05T13:20:20.012888+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:20.012888+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13030,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16975","reported_seq":17120,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.055811+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:20.055811+0000","last_peered":"2024-09-05T13:20:20.055811+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:20.055811+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17273315,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13643,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":276263419,"num_objects":67449,"num_object_clones":0,"num_object_copies":134898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223738,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9545,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":772153344,"data_stored":772072940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608112,"kb_used_data":279328,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93763728,"statfs":{"total":96636764160,"available":96014057472,"internally_reserved":0,"allocated":286031872,"data_stored":285761870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705896,"kb_used_data":273048,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93665944,"statfs":{"total":96636764160,"available":95913926656,"internally_reserved":0,"allocated":279601152,"data_stored":279331145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557916,"kb_used_data":205512,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93813924,"statfs":{"total":96636764160,"available":96065458176,"internally_reserved":0,"allocated":210444288,"data_stored":210108788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278487040,"data_stored":278483427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208842752,"data_stored":208773646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284823552,"data_stored":284815867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:23.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:24.143 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:24.144 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:24.170 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-05T13:20:23.694967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276263438,"num_objects":67451,"num_object_clones":0,"num_object_copies":134902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223740,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9546,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160723,"ondisk_log_size":160723,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871924,"kb_used_data":757888,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281243596,"statfs":{"total":289910292480,"available":287993442304,"internally_reserved":0,"allocated":776077312,"data_stored":775201803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1140706310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,6,3,11],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18989056,"num_objects":-4636,"num_object_clones":0,"num_object_copies":-9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4636,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004853"},"pg_stats":[{"pgid":"2.f","version":"47'16711","reported_seq":16791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.055412+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:20.055412+0000","last_peered":"2024-09-05T13:20:20.055412+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:20.055412+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17453056,"num_objects":4261,"num_object_clones":0,"num_object_copies":8522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16711,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-160}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16605","reported_seq":16715,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.601320+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:18.601320+0000","last_peered":"2024-09-05T13:20:18.601320+0000","last_clean":"2024-09-05T13:20:18.601320+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:18.601320+0000","last_undegraded":"2024-09-05T13:20:18.601320+0000","last_fullsized":"2024-09-05T13:20:18.601320+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14615,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16873","reported_seq":18438,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.685211+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:18.685211+0000","last_peered":"2024-09-05T13:20:18.685211+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:18.685211+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17702936,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3778,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14684,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16819","reported_seq":16894,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.667010+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:18.667010+0000","last_peered":"2024-09-05T13:20:18.667010+0000","last_clean":"2024-09-05T13:20:18.667010+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:18.667010+0000","last_undegraded":"2024-09-05T13:20:18.667010+0000","last_fullsized":"2024-09-05T13:20:18.667010+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16819,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16756","reported_seq":16896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:20.033862+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:20.033862+0000","last_peered":"2024-09-05T13:20:20.033862+0000","last_clean":"2024-09-05T13:20:20.033862+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:20.033862+0000","last_undegraded":"2024-09-05T13:20:20.033862+0000","last_fullsized":"2024-09-05T13:20:20.033862+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16904192,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13296,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16884","reported_seq":17028,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.026383+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:20.026383+0000","last_peered":"2024-09-05T13:20:20.026383+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:20.026383+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13339,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16513","reported_seq":25544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.884533+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:18.884533+0000","last_peered":"2024-09-05T13:20:18.884533+0000","last_clean":"2024-09-05T13:20:18.884533+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:18.884533+0000","last_undegraded":"2024-09-05T13:20:18.884533+0000","last_fullsized":"2024-09-05T13:20:18.884533+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16855040,"num_objects":4115,"num_object_clones":0,"num_object_copies":8230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12832,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16916","reported_seq":17053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:19.884802+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:19.884802+0000","last_peered":"2024-09-05T13:20:19.884802+0000","last_clean":"2024-09-05T13:20:19.884802+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:19.884802+0000","last_undegraded":"2024-09-05T13:20:19.884802+0000","last_fullsized":"2024-09-05T13:20:19.884802+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17154048,"num_objects":4188,"num_object_clones":0,"num_object_copies":8376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16795","reported_seq":16935,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.604761+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:18.604761+0000","last_peered":"2024-09-05T13:20:18.604761+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:18.604761+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14449,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-153}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16714","reported_seq":16857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.058064+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:20.058064+0000","last_peered":"2024-09-05T13:20:20.058064+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:20.058064+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17186816,"num_objects":4196,"num_object_clones":0,"num_object_copies":8392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13081,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16620","reported_seq":25526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.438525+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:18.438525+0000","last_peered":"2024-09-05T13:20:18.438525+0000","last_clean":"2024-09-05T13:20:18.438525+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:18.438525+0000","last_undegraded":"2024-09-05T13:20:18.438525+0000","last_fullsized":"2024-09-05T13:20:18.438525+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16432","reported_seq":16560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.608437+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:18.608437+0000","last_peered":"2024-09-05T13:20:18.608437+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:18.608437+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16998400,"num_objects":4150,"num_object_clones":0,"num_object_copies":8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13941,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-237}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16686","reported_seq":16832,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:19.947213+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:19.947213+0000","last_peered":"2024-09-05T13:20:19.947213+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:19.947213+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12837,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-227}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16821","reported_seq":17014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.056438+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:20.056438+0000","last_peered":"2024-09-05T13:20:20.056438+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:20.056438+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543168,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12720,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16900","reported_seq":17037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.012888+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:20.012888+0000","last_peered":"2024-09-05T13:20:20.012888+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:20.012888+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13030,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16975","reported_seq":17120,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.055811+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:20.055811+0000","last_peered":"2024-09-05T13:20:20.055811+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:20.055811+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17273315,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13643,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":276263419,"num_objects":67449,"num_object_clones":0,"num_object_copies":134898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223738,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9545,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":772153344,"data_stored":772072940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608112,"kb_used_data":279328,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93763728,"statfs":{"total":96636764160,"available":96014057472,"internally_reserved":0,"allocated":286031872,"data_stored":285761870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705896,"kb_used_data":273048,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93665944,"statfs":{"total":96636764160,"available":95913926656,"internally_reserved":0,"allocated":279601152,"data_stored":279331145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557916,"kb_used_data":205512,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93813924,"statfs":{"total":96636764160,"available":96065458176,"internally_reserved":0,"allocated":210444288,"data_stored":210108788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278487040,"data_stored":278483427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208842752,"data_stored":208773646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284823552,"data_stored":284815867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:24.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:24.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16711", 'reported_seq': 16791, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:20.055412+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:20.055412+0000', 'last_peered': '2024-09-05T13:20:20.055412+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:20.055412+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17453056, 'num_objects': 4261, 'num_object_clones': 0, 'num_object_copies': 8522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16711, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -160}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16873", 'reported_seq': 18438, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:18.685211+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:18.685211+0000', 'last_peered': '2024-09-05T13:20:18.685211+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:18.685211+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17702936, 'num_objects': 4323, 'num_object_clones': 0, 'num_object_copies': 8646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3778, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4323, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14684, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 715, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 545}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16884", 'reported_seq': 17028, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:20.026383+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:20.026383+0000', 'last_peered': '2024-09-05T13:20:20.026383+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:20.026383+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17494016, 'num_objects': 4271, 'num_object_clones': 0, 'num_object_copies': 8542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13339, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -180}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16795", 'reported_seq': 16935, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:18.604761+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:18.604761+0000', 'last_peered': '2024-09-05T13:20:18.604761+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:18.604761+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17604608, 'num_objects': 4298, 'num_object_clones': 0, 'num_object_copies': 8596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14449, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -153}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16714", 'reported_seq': 16857, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:20.058064+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:20.058064+0000', 'last_peered': '2024-09-05T13:20:20.058064+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:20.058064+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17186816, 'num_objects': 4196, 'num_object_clones': 0, 'num_object_copies': 8392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4196, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13081, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:24.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'16432", 'reported_seq': 16560, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:18.608437+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:18.608437+0000', 'last_peered': '2024-09-05T13:20:18.608437+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:18.608437+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16998400, 'num_objects': 4150, 'num_object_clones': 0, 'num_object_copies': 8300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13941, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -237}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:24.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:24.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16686", 'reported_seq': 16832, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:19.947213+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:19.947213+0000', 'last_peered': '2024-09-05T13:20:19.947213+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:19.947213+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17170432, 'num_objects': 4192, 'num_object_clones': 0, 'num_object_copies': 8384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4192, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12837, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -227}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:24.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:24.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16821", 'reported_seq': 17014, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:20.056438+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:20.056438+0000', 'last_peered': '2024-09-05T13:20:20.056438+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:20.056438+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17543168, 'num_objects': 4283, 'num_object_clones': 0, 'num_object_copies': 8566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12720, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:24.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:24.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16900", 'reported_seq': 17037, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:20.012888+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:20.012888+0000', 'last_peered': '2024-09-05T13:20:20.012888+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:20.012888+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17121280, 'num_objects': 4180, 'num_object_clones': 0, 'num_object_copies': 8360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13030, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -386}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:24.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:24.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16975", 'reported_seq': 17120, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:20.055811+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:20.055811+0000', 'last_peered': '2024-09-05T13:20:20.055811+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:20.055811+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17273315, 'num_objects': 4218, 'num_object_clones': 0, 'num_object_copies': 8436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4218, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13643, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -347}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:24.174 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:24.863 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:24.863 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43953/134902 objects degraded (32.581%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":551,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":67451,"data_bytes":276263438,"bytes_used":1916850176,"bytes_avail":287993442304,"bytes_total":289910292480,"degraded_objects":43953,"degraded_total":134902,"degraded_ratio":0.32581429482142599,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":158,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:24.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:24.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:25.442 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:25.443 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:25.469 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-05T13:20:23.694967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":276263438,"num_objects":67451,"num_object_clones":0,"num_object_copies":134902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223740,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9546,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160723,"ondisk_log_size":160723,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871924,"kb_used_data":757888,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281243596,"statfs":{"total":289910292480,"available":287993442304,"internally_reserved":0,"allocated":776077312,"data_stored":775201803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1140706310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,6,3,11],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18989056,"num_objects":-4636,"num_object_clones":0,"num_object_copies":-9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4636,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004853"},"pg_stats":[{"pgid":"2.f","version":"47'16711","reported_seq":16791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.055412+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:20.055412+0000","last_peered":"2024-09-05T13:20:20.055412+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:20.055412+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17453056,"num_objects":4261,"num_object_clones":0,"num_object_copies":8522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16711,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-160}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16605","reported_seq":16715,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.601320+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:18.601320+0000","last_peered":"2024-09-05T13:20:18.601320+0000","last_clean":"2024-09-05T13:20:18.601320+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:18.601320+0000","last_undegraded":"2024-09-05T13:20:18.601320+0000","last_fullsized":"2024-09-05T13:20:18.601320+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14615,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16873","reported_seq":18438,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.685211+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:18.685211+0000","last_peered":"2024-09-05T13:20:18.685211+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:18.685211+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17702936,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3778,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14684,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16819","reported_seq":16894,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.667010+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:18.667010+0000","last_peered":"2024-09-05T13:20:18.667010+0000","last_clean":"2024-09-05T13:20:18.667010+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:18.667010+0000","last_undegraded":"2024-09-05T13:20:18.667010+0000","last_fullsized":"2024-09-05T13:20:18.667010+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16819,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16756","reported_seq":16896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:20.033862+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:20.033862+0000","last_peered":"2024-09-05T13:20:20.033862+0000","last_clean":"2024-09-05T13:20:20.033862+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:20.033862+0000","last_undegraded":"2024-09-05T13:20:20.033862+0000","last_fullsized":"2024-09-05T13:20:20.033862+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16904192,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13296,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16884","reported_seq":17028,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.026383+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:20.026383+0000","last_peered":"2024-09-05T13:20:20.026383+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:20.026383+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17494016,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13339,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16513","reported_seq":25544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.884533+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:18.884533+0000","last_peered":"2024-09-05T13:20:18.884533+0000","last_clean":"2024-09-05T13:20:18.884533+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:18.884533+0000","last_undegraded":"2024-09-05T13:20:18.884533+0000","last_fullsized":"2024-09-05T13:20:18.884533+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16855040,"num_objects":4115,"num_object_clones":0,"num_object_copies":8230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12832,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'16916","reported_seq":17053,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:19.884802+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:19.884802+0000","last_peered":"2024-09-05T13:20:19.884802+0000","last_clean":"2024-09-05T13:20:19.884802+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:19.884802+0000","last_undegraded":"2024-09-05T13:20:19.884802+0000","last_fullsized":"2024-09-05T13:20:19.884802+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17154048,"num_objects":4188,"num_object_clones":0,"num_object_copies":8376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16795","reported_seq":16935,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.604761+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:18.604761+0000","last_peered":"2024-09-05T13:20:18.604761+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:18.604761+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14449,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-153}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16714","reported_seq":16857,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.058064+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:20.058064+0000","last_peered":"2024-09-05T13:20:20.058064+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:20.058064+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17186816,"num_objects":4196,"num_object_clones":0,"num_object_copies":8392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13081,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106759+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106759+0000","last_peered":"2024-09-05T13:19:36.106759+0000","last_clean":"2024-09-05T13:19:36.106759+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106759+0000","last_undegraded":"2024-09-05T13:19:36.106759+0000","last_fullsized":"2024-09-05T13:19:36.106759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16620","reported_seq":25526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:18.438525+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:18.438525+0000","last_peered":"2024-09-05T13:20:18.438525+0000","last_clean":"2024-09-05T13:20:18.438525+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:18.438525+0000","last_undegraded":"2024-09-05T13:20:18.438525+0000","last_fullsized":"2024-09-05T13:20:18.438525+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16432","reported_seq":16560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:18.608437+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:18.608437+0000","last_peered":"2024-09-05T13:20:18.608437+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:18.608437+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16998400,"num_objects":4150,"num_object_clones":0,"num_object_copies":8300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13941,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-237}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16686","reported_seq":16832,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:19.947213+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:19.947213+0000","last_peered":"2024-09-05T13:20:19.947213+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:19.947213+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12837,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-227}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16821","reported_seq":17014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.056438+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:20.056438+0000","last_peered":"2024-09-05T13:20:20.056438+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:20.056438+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543168,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12720,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16900","reported_seq":17037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.012888+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:20.012888+0000","last_peered":"2024-09-05T13:20:20.012888+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:20.012888+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17121280,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13030,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16975","reported_seq":17120,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:20.055811+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:20.055811+0000","last_peered":"2024-09-05T13:20:20.055811+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:20.055811+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17273315,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13643,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":276263419,"num_objects":67449,"num_object_clones":0,"num_object_copies":134898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223738,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9545,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":772153344,"data_stored":772072940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608112,"kb_used_data":279328,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93763728,"statfs":{"total":96636764160,"available":96014057472,"internally_reserved":0,"allocated":286031872,"data_stored":285761870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705896,"kb_used_data":273048,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93665944,"statfs":{"total":96636764160,"available":95913926656,"internally_reserved":0,"allocated":279601152,"data_stored":279331145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557916,"kb_used_data":205512,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93813924,"statfs":{"total":96636764160,"available":96065458176,"internally_reserved":0,"allocated":210444288,"data_stored":210108788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278487040,"data_stored":278483427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208842752,"data_stored":208773646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":284823552,"data_stored":284815867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:28.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:29.037 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:29.063 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-05T13:20:27.696574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266723854,"num_objects":65122,"num_object_clones":0,"num_object_copies":130244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226069,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9631,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857488,"kb_used_data":739296,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281258032,"statfs":{"total":289910292480,"available":288008224768,"internally_reserved":0,"allocated":757039104,"data_stored":756163595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1144966150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,10,8,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18993152,"num_objects":-4637,"num_object_clones":0,"num_object_copies":-9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4637,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004801"},"pg_stats":[{"pgid":"2.f","version":"47'16866","reported_seq":16946,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.073552+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:25.073552+0000","last_peered":"2024-09-05T13:20:25.073552+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:25.073552+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16818176,"num_objects":4106,"num_object_clones":0,"num_object_copies":8212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16866,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-315}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16752","reported_seq":16862,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:23.664287+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:23.664287+0000","last_peered":"2024-09-05T13:20:23.664287+0000","last_clean":"2024-09-05T13:20:23.664287+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:23.664287+0000","last_undegraded":"2024-09-05T13:20:23.664287+0000","last_fullsized":"2024-09-05T13:20:23.664287+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392192,"num_objects":4002,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14762,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17014","reported_seq":18749,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:23.609191+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:23.609191+0000","last_peered":"2024-09-05T13:20:23.609191+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:23.609191+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125400,"num_objects":4182,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14825,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":489}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16967","reported_seq":17042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:23.635676+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:23.635676+0000","last_peered":"2024-09-05T13:20:23.635676+0000","last_clean":"2024-09-05T13:20:23.635676+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:23.635676+0000","last_undegraded":"2024-09-05T13:20:23.635676+0000","last_fullsized":"2024-09-05T13:20:23.635676+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16625664,"num_objects":4059,"num_object_clones":0,"num_object_copies":8118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16967,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16895","reported_seq":17034,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.133441+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:25.133441+0000","last_peered":"2024-09-05T13:20:25.133441+0000","last_clean":"2024-09-05T13:20:25.133441+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:25.133441+0000","last_undegraded":"2024-09-05T13:20:25.133441+0000","last_fullsized":"2024-09-05T13:20:25.133441+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13435,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17025","reported_seq":17168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.116011+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:25.116011+0000","last_peered":"2024-09-05T13:20:25.116011+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:25.116011+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16916480,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13480,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16649","reported_seq":25681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:23.920901+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:23.920901+0000","last_peered":"2024-09-05T13:20:23.920901+0000","last_clean":"2024-09-05T13:20:23.920901+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:23.920901+0000","last_undegraded":"2024-09-05T13:20:23.920901+0000","last_fullsized":"2024-09-05T13:20:23.920901+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16297984,"num_objects":3979,"num_object_clones":0,"num_object_copies":7958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12968,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17052","reported_seq":17189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.034913+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:25.034913+0000","last_peered":"2024-09-05T13:20:25.034913+0000","last_clean":"2024-09-05T13:20:25.034913+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:25.034913+0000","last_undegraded":"2024-09-05T13:20:25.034913+0000","last_fullsized":"2024-09-05T13:20:25.034913+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16596992,"num_objects":4052,"num_object_clones":0,"num_object_copies":8104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13662,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16948","reported_seq":17084,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:23.657378+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:23.657378+0000","last_peered":"2024-09-05T13:20:23.657378+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:23.657378+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14602,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16850","reported_seq":16993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.114996+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:25.114996+0000","last_peered":"2024-09-05T13:20:25.114996+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:25.114996+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16629760,"num_objects":4060,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13217,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-394}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16772","reported_seq":25678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:23.608577+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:23.608577+0000","last_peered":"2024-09-05T13:20:23.608577+0000","last_clean":"2024-09-05T13:20:23.608577+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:23.608577+0000","last_undegraded":"2024-09-05T13:20:23.608577+0000","last_fullsized":"2024-09-05T13:20:23.608577+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953344,"num_objects":4139,"num_object_clones":0,"num_object_copies":8278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14367,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16563","reported_seq":16692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:23.622382+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:23.622382+0000","last_peered":"2024-09-05T13:20:23.622382+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:23.622382+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16461824,"num_objects":4019,"num_object_clones":0,"num_object_copies":8038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14072,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-368}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16834","reported_seq":16980,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.158650+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:25.158650+0000","last_peered":"2024-09-05T13:20:25.158650+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:25.158650+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12985,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16986","reported_seq":17179,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.156899+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:25.156899+0000","last_peered":"2024-09-05T13:20:25.156899+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:25.156899+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16867328,"num_objects":4118,"num_object_clones":0,"num_object_copies":8236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12885,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-326}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17036","reported_seq":17180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.059086+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:25.059086+0000","last_peered":"2024-09-05T13:20:25.059086+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:25.059086+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17140","reported_seq":17285,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.099736+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:25.099736+0000","last_peered":"2024-09-05T13:20:25.099736+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:25.099736+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16597475,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13808,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":266723835,"num_objects":65120,"num_object_clones":0,"num_object_copies":130240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65120,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226067,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9630,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753119232,"data_stored":753038828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595188,"kb_used_data":274724,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93776652,"statfs":{"total":96636764160,"available":96027291648,"internally_reserved":0,"allocated":281317376,"data_stored":281047374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":328134317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703072,"kb_used_data":266064,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93668768,"statfs":{"total":96636764160,"available":95916818432,"internally_reserved":0,"allocated":272449536,"data_stored":272179529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559228,"kb_used_data":198508,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93812612,"statfs":{"total":96636764160,"available":96064114688,"internally_reserved":0,"allocated":203272192,"data_stored":202936692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271314944,"data_stored":271311331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":201613312,"data_stored":201544206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280190976,"data_stored":280183291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:29.064 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:29.638 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:29.663 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-05T13:20:27.696574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":266723854,"num_objects":65122,"num_object_clones":0,"num_object_copies":130244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226069,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9631,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857488,"kb_used_data":739296,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281258032,"statfs":{"total":289910292480,"available":288008224768,"internally_reserved":0,"allocated":757039104,"data_stored":756163595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1144966150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,10,8,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18993152,"num_objects":-4637,"num_object_clones":0,"num_object_copies":-9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4637,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004801"},"pg_stats":[{"pgid":"2.f","version":"47'16866","reported_seq":16946,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.073552+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:25.073552+0000","last_peered":"2024-09-05T13:20:25.073552+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:25.073552+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16818176,"num_objects":4106,"num_object_clones":0,"num_object_copies":8212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16866,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-315}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16752","reported_seq":16862,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:23.664287+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:23.664287+0000","last_peered":"2024-09-05T13:20:23.664287+0000","last_clean":"2024-09-05T13:20:23.664287+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:23.664287+0000","last_undegraded":"2024-09-05T13:20:23.664287+0000","last_fullsized":"2024-09-05T13:20:23.664287+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392192,"num_objects":4002,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14762,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17014","reported_seq":18749,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:23.609191+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:23.609191+0000","last_peered":"2024-09-05T13:20:23.609191+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:23.609191+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125400,"num_objects":4182,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14825,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":489}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16967","reported_seq":17042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:23.635676+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:23.635676+0000","last_peered":"2024-09-05T13:20:23.635676+0000","last_clean":"2024-09-05T13:20:23.635676+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:23.635676+0000","last_undegraded":"2024-09-05T13:20:23.635676+0000","last_fullsized":"2024-09-05T13:20:23.635676+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16625664,"num_objects":4059,"num_object_clones":0,"num_object_copies":8118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16967,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16895","reported_seq":17034,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.133441+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:25.133441+0000","last_peered":"2024-09-05T13:20:25.133441+0000","last_clean":"2024-09-05T13:20:25.133441+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:25.133441+0000","last_undegraded":"2024-09-05T13:20:25.133441+0000","last_fullsized":"2024-09-05T13:20:25.133441+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13435,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17025","reported_seq":17168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.116011+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:25.116011+0000","last_peered":"2024-09-05T13:20:25.116011+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:25.116011+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16916480,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13480,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16649","reported_seq":25681,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:23.920901+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:23.920901+0000","last_peered":"2024-09-05T13:20:23.920901+0000","last_clean":"2024-09-05T13:20:23.920901+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:23.920901+0000","last_undegraded":"2024-09-05T13:20:23.920901+0000","last_fullsized":"2024-09-05T13:20:23.920901+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16297984,"num_objects":3979,"num_object_clones":0,"num_object_copies":7958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12968,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17052","reported_seq":17189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.034913+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:25.034913+0000","last_peered":"2024-09-05T13:20:25.034913+0000","last_clean":"2024-09-05T13:20:25.034913+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:25.034913+0000","last_undegraded":"2024-09-05T13:20:25.034913+0000","last_fullsized":"2024-09-05T13:20:25.034913+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16596992,"num_objects":4052,"num_object_clones":0,"num_object_copies":8104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13662,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16948","reported_seq":17084,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:23.657378+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:23.657378+0000","last_peered":"2024-09-05T13:20:23.657378+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:23.657378+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16977920,"num_objects":4145,"num_object_clones":0,"num_object_copies":8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14602,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16850","reported_seq":16993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.114996+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:25.114996+0000","last_peered":"2024-09-05T13:20:25.114996+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:25.114996+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16629760,"num_objects":4060,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13217,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-394}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16772","reported_seq":25678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:23.608577+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:23.608577+0000","last_peered":"2024-09-05T13:20:23.608577+0000","last_clean":"2024-09-05T13:20:23.608577+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:23.608577+0000","last_undegraded":"2024-09-05T13:20:23.608577+0000","last_fullsized":"2024-09-05T13:20:23.608577+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953344,"num_objects":4139,"num_object_clones":0,"num_object_copies":8278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14367,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16563","reported_seq":16692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:23.622382+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:23.622382+0000","last_peered":"2024-09-05T13:20:23.622382+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:23.622382+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16461824,"num_objects":4019,"num_object_clones":0,"num_object_copies":8038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14072,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-368}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16834","reported_seq":16980,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.158650+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:25.158650+0000","last_peered":"2024-09-05T13:20:25.158650+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:25.158650+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12985,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16986","reported_seq":17179,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.156899+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:25.156899+0000","last_peered":"2024-09-05T13:20:25.156899+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:25.156899+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16867328,"num_objects":4118,"num_object_clones":0,"num_object_copies":8236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12885,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-326}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17036","reported_seq":17180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.059086+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:25.059086+0000","last_peered":"2024-09-05T13:20:25.059086+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:25.059086+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17140","reported_seq":17285,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.099736+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:25.099736+0000","last_peered":"2024-09-05T13:20:25.099736+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:25.099736+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16597475,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13808,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":266723835,"num_objects":65120,"num_object_clones":0,"num_object_copies":130240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65120,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226067,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9630,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753119232,"data_stored":753038828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595188,"kb_used_data":274724,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93776652,"statfs":{"total":96636764160,"available":96027291648,"internally_reserved":0,"allocated":281317376,"data_stored":281047374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":328134317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703072,"kb_used_data":266064,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93668768,"statfs":{"total":96636764160,"available":95916818432,"internally_reserved":0,"allocated":272449536,"data_stored":272179529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559228,"kb_used_data":198508,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93812612,"statfs":{"total":96636764160,"available":96064114688,"internally_reserved":0,"allocated":203272192,"data_stored":202936692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271314944,"data_stored":271311331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":201613312,"data_stored":201544206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":280190976,"data_stored":280183291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:29.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:29.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16866", 'reported_seq': 16946, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:25.073552+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:25.073552+0000', 'last_peered': '2024-09-05T13:20:25.073552+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:25.073552+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16818176, 'num_objects': 4106, 'num_object_clones': 0, 'num_object_copies': 8212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16866, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -315}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:29.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:29.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17014", 'reported_seq': 18749, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:23.609191+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:23.609191+0000', 'last_peered': '2024-09-05T13:20:23.609191+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:23.609191+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17125400, 'num_objects': 4182, 'num_object_clones': 0, 'num_object_copies': 8364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3693, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4182, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14825, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 800, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 489}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:29.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:29.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'17025", 'reported_seq': 17168, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:25.116011+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:25.116011+0000', 'last_peered': '2024-09-05T13:20:25.116011+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:25.116011+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16916480, 'num_objects': 4130, 'num_object_clones': 0, 'num_object_copies': 8260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13480, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16948", 'reported_seq': 17084, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:23.657378+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:23.657378+0000', 'last_peered': '2024-09-05T13:20:23.657378+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:23.657378+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16977920, 'num_objects': 4145, 'num_object_clones': 0, 'num_object_copies': 8290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14602, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -306}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16850", 'reported_seq': 16993, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:25.114996+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:25.114996+0000', 'last_peered': '2024-09-05T13:20:25.114996+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:25.114996+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16629760, 'num_objects': 4060, 'num_object_clones': 0, 'num_object_copies': 8120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4060, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13217, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -394}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'16563", 'reported_seq': 16692, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:23.622382+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:23.622382+0000', 'last_peered': '2024-09-05T13:20:23.622382+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:23.622382+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16461824, 'num_objects': 4019, 'num_object_clones': 0, 'num_object_copies': 8038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14072, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -368}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16834", 'reported_seq': 16980, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:25.158650+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:25.158650+0000', 'last_peered': '2024-09-05T13:20:25.158650+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:25.158650+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16564224, 'num_objects': 4044, 'num_object_clones': 0, 'num_object_copies': 8088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12985, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -375}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'16986", 'reported_seq': 17179, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:25.156899+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:25.156899+0000', 'last_peered': '2024-09-05T13:20:25.156899+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:25.156899+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16867328, 'num_objects': 4118, 'num_object_clones': 0, 'num_object_copies': 8236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12885, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -326}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:29.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17036", 'reported_seq': 17180, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:25.059086+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:25.059086+0000', 'last_peered': '2024-09-05T13:20:25.059086+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:25.059086+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16564224, 'num_objects': 4044, 'num_object_clones': 0, 'num_object_copies': 8088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13166, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -522}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:29.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:29.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'17140", 'reported_seq': 17285, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:25.099736+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:25.099736+0000', 'last_peered': '2024-09-05T13:20:25.099736+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:25.099736+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16597475, 'num_objects': 4053, 'num_object_clones': 0, 'num_object_copies': 8106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4053, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13808, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:29.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:30.340 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43868/130244 objects degraded (33.681%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":557,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65122,"data_bytes":266723854,"bytes_used":1902067712,"bytes_avail":288008224768,"bytes_total":289910292480,"degraded_objects":43868,"degraded_total":130244,"degraded_ratio":0.33681397991462186,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":167,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:30.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:30.359 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:30.919 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:30.921 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:30.944 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-05T13:20:29.697340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":262451726,"num_objects":64079,"num_object_clones":0,"num_object_copies":128158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227112,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9711,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1858096,"kb_used_data":727424,"kb_used_omap":12,"kb_used_meta":1130611,"kb_avail":281257424,"statfs":{"total":289910292480,"available":288007602176,"internally_reserved":0,"allocated":744882176,"data_stored":744006667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1157745670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,11,8,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23265280,"num_objects":-5680,"num_object_clones":0,"num_object_copies":-11360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5680,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004792"},"pg_stats":[{"pgid":"2.f","version":"47'16866","reported_seq":16946,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.073552+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:25.073552+0000","last_peered":"2024-09-05T13:20:25.073552+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:25.073552+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16818176,"num_objects":4106,"num_object_clones":0,"num_object_copies":8212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16866,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-315}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16894","reported_seq":17004,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.663307+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:28.663307+0000","last_peered":"2024-09-05T13:20:28.663307+0000","last_clean":"2024-09-05T13:20:28.663307+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:28.663307+0000","last_undegraded":"2024-09-05T13:20:28.663307+0000","last_fullsized":"2024-09-05T13:20:28.663307+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15810560,"num_objects":3860,"num_object_clones":0,"num_object_copies":7720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14904,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17177","reported_seq":19072,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.633913+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:28.633913+0000","last_peered":"2024-09-05T13:20:28.633913+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:28.633913+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457752,"num_objects":4019,"num_object_clones":0,"num_object_copies":8038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4019,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14988,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":880,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17100","reported_seq":17175,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.568751+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:28.568751+0000","last_peered":"2024-09-05T13:20:28.568751+0000","last_clean":"2024-09-05T13:20:28.568751+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:28.568751+0000","last_undegraded":"2024-09-05T13:20:28.568751+0000","last_fullsized":"2024-09-05T13:20:28.568751+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17100,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16895","reported_seq":17034,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.133441+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:25.133441+0000","last_peered":"2024-09-05T13:20:25.133441+0000","last_clean":"2024-09-05T13:20:25.133441+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:25.133441+0000","last_undegraded":"2024-09-05T13:20:25.133441+0000","last_fullsized":"2024-09-05T13:20:25.133441+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16334848,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13435,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17025","reported_seq":17168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.116011+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:25.116011+0000","last_peered":"2024-09-05T13:20:25.116011+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:25.116011+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16916480,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13480,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16819","reported_seq":25851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:29.000939+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:29.000939+0000","last_peered":"2024-09-05T13:20:29.000939+0000","last_clean":"2024-09-05T13:20:29.000939+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:29.000939+0000","last_undegraded":"2024-09-05T13:20:29.000939+0000","last_fullsized":"2024-09-05T13:20:29.000939+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15601664,"num_objects":3809,"num_object_clones":0,"num_object_copies":7618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13138,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17052","reported_seq":17189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.034913+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:25.034913+0000","last_peered":"2024-09-05T13:20:25.034913+0000","last_clean":"2024-09-05T13:20:25.034913+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:25.034913+0000","last_undegraded":"2024-09-05T13:20:25.034913+0000","last_fullsized":"2024-09-05T13:20:25.034913+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16596992,"num_objects":4052,"num_object_clones":0,"num_object_copies":8104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13662,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17087","reported_seq":17227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.633609+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:28.633609+0000","last_peered":"2024-09-05T13:20:28.633609+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:28.633609+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16408576,"num_objects":4006,"num_object_clones":0,"num_object_copies":8012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14741,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-445}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16850","reported_seq":16993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.114996+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:25.114996+0000","last_peered":"2024-09-05T13:20:25.114996+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:25.114996+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16629760,"num_objects":4060,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13217,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-394}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16918","reported_seq":25824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.596228+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:28.596228+0000","last_peered":"2024-09-05T13:20:28.596228+0000","last_clean":"2024-09-05T13:20:28.596228+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:28.596228+0000","last_undegraded":"2024-09-05T13:20:28.596228+0000","last_fullsized":"2024-09-05T13:20:28.596228+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16355328,"num_objects":3993,"num_object_clones":0,"num_object_copies":7986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14513,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16713","reported_seq":16842,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.599763+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:28.599763+0000","last_peered":"2024-09-05T13:20:28.599763+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:28.599763+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15847424,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14222,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-518}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16834","reported_seq":16980,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.158650+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:25.158650+0000","last_peered":"2024-09-05T13:20:25.158650+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:25.158650+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12985,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-375}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16986","reported_seq":17179,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.156899+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:25.156899+0000","last_peered":"2024-09-05T13:20:25.156899+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:25.156899+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16867328,"num_objects":4118,"num_object_clones":0,"num_object_copies":8236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12885,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-326}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17036","reported_seq":17180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.059086+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:25.059086+0000","last_peered":"2024-09-05T13:20:25.059086+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:25.059086+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13166,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17140","reported_seq":17285,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:25.099736+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:25.099736+0000","last_peered":"2024-09-05T13:20:25.099736+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:25.099736+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16597475,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13808,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":262451707,"num_objects":64077,"num_object_clones":0,"num_object_copies":128154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227110,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9710,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":741179392,"data_stored":741098988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594784,"kb_used_data":270160,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93777056,"statfs":{"total":96636764160,"available":96027705344,"internally_reserved":0,"allocated":276643840,"data_stored":276373838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703072,"kb_used_data":266064,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93668768,"statfs":{"total":96636764160,"available":95916818432,"internally_reserved":0,"allocated":272449536,"data_stored":272179529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560240,"kb_used_data":191200,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93811600,"statfs":{"total":96636764160,"available":96063078400,"internally_reserved":0,"allocated":195788800,"data_stored":195453300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":377876143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271314944,"data_stored":271311331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":194310144,"data_stored":194241038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275554304,"data_stored":275546619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:33.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:34.487 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:34.489 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:34.509 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-05T13:20:33.698964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257151502,"num_objects":62785,"num_object_clones":0,"num_object_copies":125570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":228406,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9711,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160989,"ondisk_log_size":160989,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1842860,"kb_used_data":720504,"kb_used_omap":12,"kb_used_meta":1122291,"kb_avail":281272660,"statfs":{"total":289910292480,"available":288023203840,"internally_reserved":0,"allocated":737796096,"data_stored":736920587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1149225990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,7,9,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19111936,"num_objects":-4666,"num_object_clones":0,"num_object_copies":-9332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004827"},"pg_stats":[{"pgid":"2.f","version":"47'17010","reported_seq":17089,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.144678+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:30.144678+0000","last_peered":"2024-09-05T13:20:30.144678+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:30.144678+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17010,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-459}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16894","reported_seq":17004,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.663307+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:28.663307+0000","last_peered":"2024-09-05T13:20:28.663307+0000","last_clean":"2024-09-05T13:20:28.663307+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:28.663307+0000","last_undegraded":"2024-09-05T13:20:28.663307+0000","last_fullsized":"2024-09-05T13:20:28.663307+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15810560,"num_objects":3860,"num_object_clones":0,"num_object_copies":7720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14904,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17177","reported_seq":19072,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.633913+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:28.633913+0000","last_peered":"2024-09-05T13:20:28.633913+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:28.633913+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457752,"num_objects":4019,"num_object_clones":0,"num_object_copies":8038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4019,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14988,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":880,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17100","reported_seq":17175,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.568751+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:28.568751+0000","last_peered":"2024-09-05T13:20:28.568751+0000","last_clean":"2024-09-05T13:20:28.568751+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:28.568751+0000","last_undegraded":"2024-09-05T13:20:28.568751+0000","last_fullsized":"2024-09-05T13:20:28.568751+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17100,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17026","reported_seq":17166,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:30.159351+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:30.159351+0000","last_peered":"2024-09-05T13:20:30.159351+0000","last_clean":"2024-09-05T13:20:30.159351+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:30.159351+0000","last_undegraded":"2024-09-05T13:20:30.159351+0000","last_fullsized":"2024-09-05T13:20:30.159351+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15798272,"num_objects":3857,"num_object_clones":0,"num_object_copies":7714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13566,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17169","reported_seq":17313,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.184333+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:30.184333+0000","last_peered":"2024-09-05T13:20:30.184333+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:30.184333+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16326656,"num_objects":3986,"num_object_clones":0,"num_object_copies":7972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13624,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16819","reported_seq":25851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:29.000939+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:29.000939+0000","last_peered":"2024-09-05T13:20:29.000939+0000","last_clean":"2024-09-05T13:20:29.000939+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:29.000939+0000","last_undegraded":"2024-09-05T13:20:29.000939+0000","last_fullsized":"2024-09-05T13:20:29.000939+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15601664,"num_objects":3809,"num_object_clones":0,"num_object_copies":7618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13138,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17184","reported_seq":17321,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:30.176664+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:30.176664+0000","last_peered":"2024-09-05T13:20:30.176664+0000","last_clean":"2024-09-05T13:20:30.176664+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:30.176664+0000","last_undegraded":"2024-09-05T13:20:30.176664+0000","last_fullsized":"2024-09-05T13:20:30.176664+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13794,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17087","reported_seq":17227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.633609+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:28.633609+0000","last_peered":"2024-09-05T13:20:28.633609+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:28.633609+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16408576,"num_objects":4006,"num_object_clones":0,"num_object_copies":8012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14741,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-445}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16989","reported_seq":17133,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.185352+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:30.185352+0000","last_peered":"2024-09-05T13:20:30.185352+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:30.185352+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13356,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-533}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16918","reported_seq":25824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.596228+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:28.596228+0000","last_peered":"2024-09-05T13:20:28.596228+0000","last_clean":"2024-09-05T13:20:28.596228+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:28.596228+0000","last_undegraded":"2024-09-05T13:20:28.596228+0000","last_fullsized":"2024-09-05T13:20:28.596228+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16355328,"num_objects":3993,"num_object_clones":0,"num_object_copies":7986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14513,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16713","reported_seq":16842,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.599763+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:28.599763+0000","last_peered":"2024-09-05T13:20:28.599763+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:28.599763+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15847424,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14222,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-518}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16969","reported_seq":17111,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.149923+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:30.149923+0000","last_peered":"2024-09-05T13:20:30.149923+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:30.149923+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13120,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17151","reported_seq":17343,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.194379+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:30.194379+0000","last_peered":"2024-09-05T13:20:30.194379+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:30.194379+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16191488,"num_objects":3953,"num_object_clones":0,"num_object_copies":7906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13050,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-491}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17181","reported_seq":17324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.172974+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:30.172974+0000","last_peered":"2024-09-05T13:20:30.172974+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:30.172974+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15970304,"num_objects":3899,"num_object_clones":0,"num_object_copies":7798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13311,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17299","reported_seq":17444,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.073477+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:30.073477+0000","last_peered":"2024-09-05T13:20:30.073477+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:30.073477+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15946211,"num_objects":3894,"num_object_clones":0,"num_object_copies":7788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257151483,"num_objects":62783,"num_object_clones":0,"num_object_copies":125566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":228404,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9710,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":734019584,"data_stored":733939180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160986,"ondisk_log_size":160986,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594784,"kb_used_data":270160,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93777056,"statfs":{"total":96636764160,"available":96027705344,"internally_reserved":0,"allocated":276643840,"data_stored":276373838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687836,"kb_used_data":259144,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93684004,"statfs":{"total":96636764160,"available":95932420096,"internally_reserved":0,"allocated":265363456,"data_stored":265093449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560240,"kb_used_data":191200,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93811600,"statfs":{"total":96636764160,"available":96063078400,"internally_reserved":0,"allocated":195788800,"data_stored":195453300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":377876143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":264155136,"data_stored":264151523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":194310144,"data_stored":194241038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275554304,"data_stored":275546619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:34.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:35.067 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:35.073 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:35.091 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-05T13:20:33.698964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":257151502,"num_objects":62785,"num_object_clones":0,"num_object_copies":125570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":228406,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9711,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160989,"ondisk_log_size":160989,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1842860,"kb_used_data":720504,"kb_used_omap":12,"kb_used_meta":1122291,"kb_avail":281272660,"statfs":{"total":289910292480,"available":288023203840,"internally_reserved":0,"allocated":737796096,"data_stored":736920587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1149225990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,7,9,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19111936,"num_objects":-4666,"num_object_clones":0,"num_object_copies":-9332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004827"},"pg_stats":[{"pgid":"2.f","version":"47'17010","reported_seq":17089,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.144678+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:30.144678+0000","last_peered":"2024-09-05T13:20:30.144678+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:30.144678+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16228352,"num_objects":3962,"num_object_clones":0,"num_object_copies":7924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17010,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-459}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16894","reported_seq":17004,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.663307+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:28.663307+0000","last_peered":"2024-09-05T13:20:28.663307+0000","last_clean":"2024-09-05T13:20:28.663307+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:28.663307+0000","last_undegraded":"2024-09-05T13:20:28.663307+0000","last_fullsized":"2024-09-05T13:20:28.663307+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15810560,"num_objects":3860,"num_object_clones":0,"num_object_copies":7720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14904,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17177","reported_seq":19072,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.633913+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:28.633913+0000","last_peered":"2024-09-05T13:20:28.633913+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:28.633913+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16457752,"num_objects":4019,"num_object_clones":0,"num_object_copies":8038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3613,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4019,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14988,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":880,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17100","reported_seq":17175,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.568751+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:28.568751+0000","last_peered":"2024-09-05T13:20:28.568751+0000","last_clean":"2024-09-05T13:20:28.568751+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:28.568751+0000","last_undegraded":"2024-09-05T13:20:28.568751+0000","last_fullsized":"2024-09-05T13:20:28.568751+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17100,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17026","reported_seq":17166,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:30.159351+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:30.159351+0000","last_peered":"2024-09-05T13:20:30.159351+0000","last_clean":"2024-09-05T13:20:30.159351+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:30.159351+0000","last_undegraded":"2024-09-05T13:20:30.159351+0000","last_fullsized":"2024-09-05T13:20:30.159351+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15798272,"num_objects":3857,"num_object_clones":0,"num_object_copies":7714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13566,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17169","reported_seq":17313,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.184333+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:30.184333+0000","last_peered":"2024-09-05T13:20:30.184333+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:30.184333+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16326656,"num_objects":3986,"num_object_clones":0,"num_object_copies":7972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13624,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16819","reported_seq":25851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:29.000939+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:29.000939+0000","last_peered":"2024-09-05T13:20:29.000939+0000","last_clean":"2024-09-05T13:20:29.000939+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:29.000939+0000","last_undegraded":"2024-09-05T13:20:29.000939+0000","last_fullsized":"2024-09-05T13:20:29.000939+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15601664,"num_objects":3809,"num_object_clones":0,"num_object_copies":7618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13138,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17184","reported_seq":17321,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:30.176664+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:30.176664+0000","last_peered":"2024-09-05T13:20:30.176664+0000","last_clean":"2024-09-05T13:20:30.176664+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:30.176664+0000","last_undegraded":"2024-09-05T13:20:30.176664+0000","last_fullsized":"2024-09-05T13:20:30.176664+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16056320,"num_objects":3920,"num_object_clones":0,"num_object_copies":7840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13794,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17087","reported_seq":17227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.633609+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:28.633609+0000","last_peered":"2024-09-05T13:20:28.633609+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:28.633609+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16408576,"num_objects":4006,"num_object_clones":0,"num_object_copies":8012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14741,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-445}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16989","reported_seq":17133,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.185352+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:30.185352+0000","last_peered":"2024-09-05T13:20:30.185352+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:30.185352+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13356,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-533}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16918","reported_seq":25824,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:28.596228+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:28.596228+0000","last_peered":"2024-09-05T13:20:28.596228+0000","last_clean":"2024-09-05T13:20:28.596228+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:28.596228+0000","last_undegraded":"2024-09-05T13:20:28.596228+0000","last_fullsized":"2024-09-05T13:20:28.596228+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16355328,"num_objects":3993,"num_object_clones":0,"num_object_copies":7986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14513,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16713","reported_seq":16842,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:28.599763+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:28.599763+0000","last_peered":"2024-09-05T13:20:28.599763+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:28.599763+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15847424,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14222,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-518}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16969","reported_seq":17111,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.149923+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:30.149923+0000","last_peered":"2024-09-05T13:20:30.149923+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:30.149923+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16011264,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13120,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17151","reported_seq":17343,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.194379+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:30.194379+0000","last_peered":"2024-09-05T13:20:30.194379+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:30.194379+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16191488,"num_objects":3953,"num_object_clones":0,"num_object_copies":7906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13050,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-491}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17181","reported_seq":17324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.172974+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:30.172974+0000","last_peered":"2024-09-05T13:20:30.172974+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:30.172974+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15970304,"num_objects":3899,"num_object_clones":0,"num_object_copies":7798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13311,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17299","reported_seq":17444,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:30.073477+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:30.073477+0000","last_peered":"2024-09-05T13:20:30.073477+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:30.073477+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15946211,"num_objects":3894,"num_object_clones":0,"num_object_copies":7788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":257151483,"num_objects":62783,"num_object_clones":0,"num_object_copies":125566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43788,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":228404,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9710,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":734019584,"data_stored":733939180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160986,"ondisk_log_size":160986,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594784,"kb_used_data":270160,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93777056,"statfs":{"total":96636764160,"available":96027705344,"internally_reserved":0,"allocated":276643840,"data_stored":276373838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687836,"kb_used_data":259144,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93684004,"statfs":{"total":96636764160,"available":95932420096,"internally_reserved":0,"allocated":265363456,"data_stored":265093449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560240,"kb_used_data":191200,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93811600,"statfs":{"total":96636764160,"available":96063078400,"internally_reserved":0,"allocated":195788800,"data_stored":195453300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":377876143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":264155136,"data_stored":264151523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":194310144,"data_stored":194241038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275554304,"data_stored":275546619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:35.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:35.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17010", 'reported_seq': 17089, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:30.144678+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:30.144678+0000', 'last_peered': '2024-09-05T13:20:30.144678+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:30.144678+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16228352, 'num_objects': 3962, 'num_object_clones': 0, 'num_object_copies': 7924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17010, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -459}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:35.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:35.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17177", 'reported_seq': 19072, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:28.633913+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:28.633913+0000', 'last_peered': '2024-09-05T13:20:28.633913+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:28.633913+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16457752, 'num_objects': 4019, 'num_object_clones': 0, 'num_object_copies': 8038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3613, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4019, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14988, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 880, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:35.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:35.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'17169", 'reported_seq': 17313, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:30.184333+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:30.184333+0000', 'last_peered': '2024-09-05T13:20:30.184333+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:30.184333+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16326656, 'num_objects': 3986, 'num_object_clones': 0, 'num_object_copies': 7972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3986, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13624, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -465}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:35.093 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17087", 'reported_seq': 17227, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:28.633609+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:28.633609+0000', 'last_peered': '2024-09-05T13:20:28.633609+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:28.633609+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16408576, 'num_objects': 4006, 'num_object_clones': 0, 'num_object_copies': 8012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14741, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -445}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16989", 'reported_seq': 17133, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:30.185352+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:30.185352+0000', 'last_peered': '2024-09-05T13:20:30.185352+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:30.185352+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16060416, 'num_objects': 3921, 'num_object_clones': 0, 'num_object_copies': 7842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3921, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13356, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -533}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'16713", 'reported_seq': 16842, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:28.599763+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:28.599763+0000', 'last_peered': '2024-09-05T13:20:28.599763+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:28.599763+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15847424, 'num_objects': 3869, 'num_object_clones': 0, 'num_object_copies': 7738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14222, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -518}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16969", 'reported_seq': 17111, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:30.149923+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:30.149923+0000', 'last_peered': '2024-09-05T13:20:30.149923+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:30.149923+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16011264, 'num_objects': 3909, 'num_object_clones': 0, 'num_object_copies': 7818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3909, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13120, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'17151", 'reported_seq': 17343, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:30.194379+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:30.194379+0000', 'last_peered': '2024-09-05T13:20:30.194379+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:30.194379+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16191488, 'num_objects': 3953, 'num_object_clones': 0, 'num_object_copies': 7906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13050, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -491}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:35.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:35.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17181", 'reported_seq': 17324, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:30.172974+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:30.172974+0000', 'last_peered': '2024-09-05T13:20:30.172974+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:30.172974+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15970304, 'num_objects': 3899, 'num_object_clones': 0, 'num_object_copies': 7798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3899, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13311, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -667}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:35.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:35.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'17299", 'reported_seq': 17444, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:30.073477+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:30.073477+0000', 'last_peered': '2024-09-05T13:20:30.073477+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:30.073477+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15946211, 'num_objects': 3894, 'num_object_clones': 0, 'num_object_copies': 7788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13967, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -671}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:35.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:35.765 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:35.765 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43709/120960 objects degraded (36.135%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":562,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":60480,"data_bytes":247710222,"bytes_used":1863872512,"bytes_avail":288046419968,"bytes_total":289910292480,"degraded_objects":43709,"degraded_total":120960,"degraded_ratio":0.36135085978835979,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":580},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:35.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:35.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:36.349 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:36.349 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:36.374 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-05T13:20:35.699693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":247710222,"num_objects":60480,"num_object_clones":0,"num_object_copies":120960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60480,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230711,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9790,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160694,"ondisk_log_size":160694,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1820188,"kb_used_data":701988,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281295332,"statfs":{"total":289910292480,"available":288046419968,"internally_reserved":0,"allocated":718835712,"data_stored":717960203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1144966150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,8,7,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28553216,"num_objects":-6971,"num_object_clones":0,"num_object_copies":-13942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004726"},"pg_stats":[{"pgid":"2.f","version":"47'17153","reported_seq":17233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.021423+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:35.021423+0000","last_peered":"2024-09-05T13:20:35.021423+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:35.021423+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15642624,"num_objects":3819,"num_object_clones":0,"num_object_copies":7638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17153,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17007","reported_seq":17117,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:33.591553+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:33.591553+0000","last_peered":"2024-09-05T13:20:33.591553+0000","last_clean":"2024-09-05T13:20:33.591553+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:33.591553+0000","last_undegraded":"2024-09-05T13:20:33.591553+0000","last_fullsized":"2024-09-05T13:20:33.591553+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15347712,"num_objects":3747,"num_object_clones":0,"num_object_copies":7494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15017,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17328","reported_seq":19381,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:33.688736+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:33.688736+0000","last_peered":"2024-09-05T13:20:33.688736+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:33.688736+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15839256,"num_objects":3868,"num_object_clones":0,"num_object_copies":7736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3534,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3868,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15139,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":959,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":334}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17265","reported_seq":17340,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:33.637321+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:33.637321+0000","last_peered":"2024-09-05T13:20:33.637321+0000","last_clean":"2024-09-05T13:20:33.637321+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:33.637321+0000","last_undegraded":"2024-09-05T13:20:33.637321+0000","last_fullsized":"2024-09-05T13:20:33.637321+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15405056,"num_objects":3761,"num_object_clones":0,"num_object_copies":7522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17265,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17184","reported_seq":17324,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:35.173502+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:35.173502+0000","last_peered":"2024-09-05T13:20:35.173502+0000","last_clean":"2024-09-05T13:20:35.173502+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:35.173502+0000","last_undegraded":"2024-09-05T13:20:35.173502+0000","last_fullsized":"2024-09-05T13:20:35.173502+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15151104,"num_objects":3699,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13724,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17316","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.137987+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:35.137987+0000","last_peered":"2024-09-05T13:20:35.137987+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:35.137987+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13771,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16959","reported_seq":25991,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:34.034031+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:34.034031+0000","last_peered":"2024-09-05T13:20:34.034031+0000","last_clean":"2024-09-05T13:20:34.034031+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:34.034031+0000","last_undegraded":"2024-09-05T13:20:34.034031+0000","last_fullsized":"2024-09-05T13:20:34.034031+0000","mapping_epoch":46,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15028224,"num_objects":3669,"num_object_clones":0,"num_object_copies":7338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13278,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17348","reported_seq":17485,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:35.176795+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:35.176795+0000","last_peered":"2024-09-05T13:20:35.176795+0000","last_clean":"2024-09-05T13:20:35.176795+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:35.176795+0000","last_undegraded":"2024-09-05T13:20:35.176795+0000","last_fullsized":"2024-09-05T13:20:35.176795+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17216","reported_seq":17355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:33.663288+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:33.663288+0000","last_peered":"2024-09-05T13:20:33.663288+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:33.663288+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15880192,"num_objects":3877,"num_object_clones":0,"num_object_copies":7754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14870,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119238+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119238+0000","last_peered":"2024-09-05T13:19:36.119238+0000","last_clean":"2024-09-05T13:19:36.119238+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119238+0000","last_undegraded":"2024-09-05T13:19:36.119238+0000","last_fullsized":"2024-09-05T13:19:36.119238+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17136","reported_seq":17280,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.163474+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:35.163474+0000","last_peered":"2024-09-05T13:20:35.163474+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:35.163474+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13503,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17058","reported_seq":25964,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:33.599273+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:33.599273+0000","last_peered":"2024-09-05T13:20:33.599273+0000","last_clean":"2024-09-05T13:20:33.599273+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:33.599273+0000","last_undegraded":"2024-09-05T13:20:33.599273+0000","last_fullsized":"2024-09-05T13:20:33.599273+0000","mapping_epoch":44,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14653,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16856","reported_seq":16985,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:33.560125+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:33.560125+0000","last_peered":"2024-09-05T13:20:33.560125+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:33.560125+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14365,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-661}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17122","reported_seq":17268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.056818+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:35.056818+0000","last_peered":"2024-09-05T13:20:35.056818+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:35.056818+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13273,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17291","reported_seq":17484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.205021+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:35.205021+0000","last_peered":"2024-09-05T13:20:35.205021+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:35.205021+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15618048,"num_objects":3813,"num_object_clones":0,"num_object_copies":7626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-631}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119937+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119937+0000","last_peered":"2024-09-05T13:19:36.119937+0000","last_clean":"2024-09-05T13:19:36.119937+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119937+0000","last_undegraded":"2024-09-05T13:19:36.119937+0000","last_fullsized":"2024-09-05T13:19:36.119937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17317","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.183019+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:35.183019+0000","last_peered":"2024-09-05T13:20:35.183019+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:35.183019+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15413248,"num_objects":3763,"num_object_clones":0,"num_object_copies":7526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13447,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17435","reported_seq":17580,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.192113+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:35.192113+0000","last_peered":"2024-09-05T13:20:35.192113+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:35.192113+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15389155,"num_objects":3758,"num_object_clones":0,"num_object_copies":7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14103,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":247710203,"num_objects":60478,"num_object_clones":0,"num_object_copies":120956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230709,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9789,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":715010048,"data_stored":714929644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160691,"ondisk_log_size":160691,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594448,"kb_used_data":265664,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93777392,"statfs":{"total":96636764160,"available":96028049408,"internally_reserved":0,"allocated":272039936,"data_stored":271769934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684908,"kb_used_data":252060,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93686932,"statfs":{"total":96636764160,"available":95935418368,"internally_reserved":0,"allocated":258109440,"data_stored":257839433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540832,"kb_used_data":184264,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93831008,"statfs":{"total":96636764160,"available":96082952192,"internally_reserved":0,"allocated":188686336,"data_stored":188350836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":256983040,"data_stored":256979427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":187097088,"data_stored":187027982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270929920,"data_stored":270922235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:39.377 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:39.928 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:39.930 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:39.957 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-05T13:20:39.701180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":243765774,"num_objects":59517,"num_object_clones":0,"num_object_copies":119034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231674,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9880,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1821492,"kb_used_data":690804,"kb_used_omap":12,"kb_used_meta":1130611,"kb_avail":281294028,"statfs":{"total":289910292480,"available":288045084672,"internally_reserved":0,"allocated":707383296,"data_stored":706507787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1157745670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,7,6,5,8,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22958080,"num_objects":-5605,"num_object_clones":0,"num_object_copies":-11210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5605,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004606"},"pg_stats":[{"pgid":"2.f","version":"47'17153","reported_seq":17233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.021423+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:35.021423+0000","last_peered":"2024-09-05T13:20:35.021423+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:35.021423+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15642624,"num_objects":3819,"num_object_clones":0,"num_object_copies":7638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17153,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17151","reported_seq":17261,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.575741+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:38.575741+0000","last_peered":"2024-09-05T13:20:38.575741+0000","last_clean":"2024-09-05T13:20:38.575741+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:38.575741+0000","last_undegraded":"2024-09-05T13:20:38.575741+0000","last_fullsized":"2024-09-05T13:20:38.575741+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14757888,"num_objects":3603,"num_object_clones":0,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15161,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17465","reported_seq":19698,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.703549+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:38.703549+0000","last_peered":"2024-09-05T13:20:38.703549+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:38.703549+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278104,"num_objects":3731,"num_object_clones":0,"num_object_copies":7462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15276,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1049,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17390","reported_seq":17465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.660914+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:38.660914+0000","last_peered":"2024-09-05T13:20:38.660914+0000","last_clean":"2024-09-05T13:20:38.660914+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:38.660914+0000","last_undegraded":"2024-09-05T13:20:38.660914+0000","last_fullsized":"2024-09-05T13:20:38.660914+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14893056,"num_objects":3636,"num_object_clones":0,"num_object_copies":7272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17390,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17184","reported_seq":17324,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:35.173502+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:35.173502+0000","last_peered":"2024-09-05T13:20:35.173502+0000","last_clean":"2024-09-05T13:20:35.173502+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:35.173502+0000","last_undegraded":"2024-09-05T13:20:35.173502+0000","last_fullsized":"2024-09-05T13:20:35.173502+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15151104,"num_objects":3699,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13724,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17316","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.137987+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:35.137987+0000","last_peered":"2024-09-05T13:20:35.137987+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:35.137987+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13771,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17104","reported_seq":26136,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:39.100584+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:39.100584+0000","last_peered":"2024-09-05T13:20:39.100584+0000","last_clean":"2024-09-05T13:20:39.100584+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:39.100584+0000","last_undegraded":"2024-09-05T13:20:39.100584+0000","last_fullsized":"2024-09-05T13:20:39.100584+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434304,"num_objects":3524,"num_object_clones":0,"num_object_copies":7048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13423,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17348","reported_seq":17485,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:35.176795+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:35.176795+0000","last_peered":"2024-09-05T13:20:35.176795+0000","last_clean":"2024-09-05T13:20:35.176795+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:35.176795+0000","last_undegraded":"2024-09-05T13:20:35.176795+0000","last_fullsized":"2024-09-05T13:20:35.176795+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17347","reported_seq":17487,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.527575+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:38.527575+0000","last_peered":"2024-09-05T13:20:38.527575+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:38.527575+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15343616,"num_objects":3746,"num_object_clones":0,"num_object_copies":7492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15001,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17136","reported_seq":17280,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.163474+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:35.163474+0000","last_peered":"2024-09-05T13:20:35.163474+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:35.163474+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13503,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17201","reported_seq":26108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.704990+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:38.704990+0000","last_peered":"2024-09-05T13:20:38.704990+0000","last_clean":"2024-09-05T13:20:38.704990+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:38.704990+0000","last_undegraded":"2024-09-05T13:20:38.704990+0000","last_fullsized":"2024-09-05T13:20:38.704990+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15196160,"num_objects":3710,"num_object_clones":0,"num_object_copies":7420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14796,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16994","reported_seq":17123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.662589+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:38.662589+0000","last_peered":"2024-09-05T13:20:38.662589+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:38.662589+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14503,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17122","reported_seq":17268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.056818+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:35.056818+0000","last_peered":"2024-09-05T13:20:35.056818+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:35.056818+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13273,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17291","reported_seq":17484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.205021+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:35.205021+0000","last_peered":"2024-09-05T13:20:35.205021+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:35.205021+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15618048,"num_objects":3813,"num_object_clones":0,"num_object_copies":7626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-631}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17317","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.183019+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:35.183019+0000","last_peered":"2024-09-05T13:20:35.183019+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:35.183019+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15413248,"num_objects":3763,"num_object_clones":0,"num_object_copies":7526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13447,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17435","reported_seq":17580,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.192113+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:35.192113+0000","last_peered":"2024-09-05T13:20:35.192113+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:35.192113+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15389155,"num_objects":3758,"num_object_clones":0,"num_object_copies":7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14103,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":243765755,"num_objects":59515,"num_object_clones":0,"num_object_copies":119030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231672,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9879,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":703344640,"data_stored":703264236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594184,"kb_used_data":261236,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93777656,"statfs":{"total":96636764160,"available":96028319744,"internally_reserved":0,"allocated":267505664,"data_stored":267235662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":340913837},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684908,"kb_used_data":252060,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93686932,"statfs":{"total":96636764160,"available":95935418368,"internally_reserved":0,"allocated":258109440,"data_stored":257839433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542400,"kb_used_data":177508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93829440,"statfs":{"total":96636764160,"available":96081346560,"internally_reserved":0,"allocated":181768192,"data_stored":181432692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":256983040,"data_stored":256979427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":179978240,"data_stored":179909134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266383360,"data_stored":266375675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:39.959 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:40.523 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:40.525 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:40.551 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-05T13:20:39.701180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":243765774,"num_objects":59517,"num_object_clones":0,"num_object_copies":119034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231674,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9880,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1821492,"kb_used_data":690804,"kb_used_omap":12,"kb_used_meta":1130611,"kb_avail":281294028,"statfs":{"total":289910292480,"available":288045084672,"internally_reserved":0,"allocated":707383296,"data_stored":706507787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1157745670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,7,6,5,8,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22958080,"num_objects":-5605,"num_object_clones":0,"num_object_copies":-11210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5605,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004606"},"pg_stats":[{"pgid":"2.f","version":"47'17153","reported_seq":17233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.021423+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:35.021423+0000","last_peered":"2024-09-05T13:20:35.021423+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:35.021423+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15642624,"num_objects":3819,"num_object_clones":0,"num_object_copies":7638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17153,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17151","reported_seq":17261,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.575741+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:38.575741+0000","last_peered":"2024-09-05T13:20:38.575741+0000","last_clean":"2024-09-05T13:20:38.575741+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:38.575741+0000","last_undegraded":"2024-09-05T13:20:38.575741+0000","last_fullsized":"2024-09-05T13:20:38.575741+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14757888,"num_objects":3603,"num_object_clones":0,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15161,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17465","reported_seq":19698,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.703549+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:38.703549+0000","last_peered":"2024-09-05T13:20:38.703549+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:38.703549+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278104,"num_objects":3731,"num_object_clones":0,"num_object_copies":7462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15276,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1049,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17390","reported_seq":17465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.660914+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:38.660914+0000","last_peered":"2024-09-05T13:20:38.660914+0000","last_clean":"2024-09-05T13:20:38.660914+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:38.660914+0000","last_undegraded":"2024-09-05T13:20:38.660914+0000","last_fullsized":"2024-09-05T13:20:38.660914+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14893056,"num_objects":3636,"num_object_clones":0,"num_object_copies":7272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17390,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17184","reported_seq":17324,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:35.173502+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:35.173502+0000","last_peered":"2024-09-05T13:20:35.173502+0000","last_clean":"2024-09-05T13:20:35.173502+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:35.173502+0000","last_undegraded":"2024-09-05T13:20:35.173502+0000","last_fullsized":"2024-09-05T13:20:35.173502+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15151104,"num_objects":3699,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13724,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17316","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.137987+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:35.137987+0000","last_peered":"2024-09-05T13:20:35.137987+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:35.137987+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13771,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17104","reported_seq":26136,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:39.100584+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:39.100584+0000","last_peered":"2024-09-05T13:20:39.100584+0000","last_clean":"2024-09-05T13:20:39.100584+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:39.100584+0000","last_undegraded":"2024-09-05T13:20:39.100584+0000","last_fullsized":"2024-09-05T13:20:39.100584+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434304,"num_objects":3524,"num_object_clones":0,"num_object_copies":7048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13423,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17348","reported_seq":17485,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:35.176795+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:35.176795+0000","last_peered":"2024-09-05T13:20:35.176795+0000","last_clean":"2024-09-05T13:20:35.176795+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:35.176795+0000","last_undegraded":"2024-09-05T13:20:35.176795+0000","last_fullsized":"2024-09-05T13:20:35.176795+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17347","reported_seq":17487,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.527575+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:38.527575+0000","last_peered":"2024-09-05T13:20:38.527575+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:38.527575+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15343616,"num_objects":3746,"num_object_clones":0,"num_object_copies":7492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15001,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17136","reported_seq":17280,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.163474+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:35.163474+0000","last_peered":"2024-09-05T13:20:35.163474+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:35.163474+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13503,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-680}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17201","reported_seq":26108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.704990+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:38.704990+0000","last_peered":"2024-09-05T13:20:38.704990+0000","last_clean":"2024-09-05T13:20:38.704990+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:38.704990+0000","last_undegraded":"2024-09-05T13:20:38.704990+0000","last_fullsized":"2024-09-05T13:20:38.704990+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15196160,"num_objects":3710,"num_object_clones":0,"num_object_copies":7420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14796,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16994","reported_seq":17123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.662589+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:38.662589+0000","last_peered":"2024-09-05T13:20:38.662589+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:38.662589+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14503,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17122","reported_seq":17268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.056818+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:35.056818+0000","last_peered":"2024-09-05T13:20:35.056818+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:35.056818+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15384576,"num_objects":3756,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13273,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17291","reported_seq":17484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.205021+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:35.205021+0000","last_peered":"2024-09-05T13:20:35.205021+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:35.205021+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15618048,"num_objects":3813,"num_object_clones":0,"num_object_copies":7626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13190,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-631}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17317","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.183019+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:35.183019+0000","last_peered":"2024-09-05T13:20:35.183019+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:35.183019+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15413248,"num_objects":3763,"num_object_clones":0,"num_object_copies":7526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13447,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17435","reported_seq":17580,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:35.192113+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:35.192113+0000","last_peered":"2024-09-05T13:20:35.192113+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:35.192113+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15389155,"num_objects":3758,"num_object_clones":0,"num_object_copies":7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14103,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":243765755,"num_objects":59515,"num_object_clones":0,"num_object_copies":119030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":231672,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9879,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":703344640,"data_stored":703264236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594184,"kb_used_data":261236,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93777656,"statfs":{"total":96636764160,"available":96028319744,"internally_reserved":0,"allocated":267505664,"data_stored":267235662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":340913837},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684908,"kb_used_data":252060,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93686932,"statfs":{"total":96636764160,"available":95935418368,"internally_reserved":0,"allocated":258109440,"data_stored":257839433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":443215530},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542400,"kb_used_data":177508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93829440,"statfs":{"total":96636764160,"available":96081346560,"internally_reserved":0,"allocated":181768192,"data_stored":181432692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":256983040,"data_stored":256979427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":179978240,"data_stored":179909134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266383360,"data_stored":266375675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:40.552 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:40.552 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17153", 'reported_seq': 17233, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:35.021423+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:35.021423+0000', 'last_peered': '2024-09-05T13:20:35.021423+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:35.021423+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15642624, 'num_objects': 3819, 'num_object_clones': 0, 'num_object_copies': 7638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3819, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17153, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -602}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17465", 'reported_seq': 19698, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:38.703549+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:38.703549+0000', 'last_peered': '2024-09-05T13:20:38.703549+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:38.703549+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15278104, 'num_objects': 3731, 'num_object_clones': 0, 'num_object_copies': 7462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3731, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15276, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1049, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 287}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'17316", 'reported_seq': 17460, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:35.137987+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:35.137987+0000', 'last_peered': '2024-09-05T13:20:35.137987+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:35.137987+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15724544, 'num_objects': 3839, 'num_object_clones': 0, 'num_object_copies': 7678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13771, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -612}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17347", 'reported_seq': 17487, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:38.527575+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:38.527575+0000', 'last_peered': '2024-09-05T13:20:38.527575+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:38.527575+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15343616, 'num_objects': 3746, 'num_object_clones': 0, 'num_object_copies': 7492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3746, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15001, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -705}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17136", 'reported_seq': 17280, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:35.163474+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:35.163474+0000', 'last_peered': '2024-09-05T13:20:35.163474+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:35.163474+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15458304, 'num_objects': 3774, 'num_object_clones': 0, 'num_object_copies': 7548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3774, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13503, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -680}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:40.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'16994", 'reported_seq': 17123, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:38.662589+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:38.662589+0000', 'last_peered': '2024-09-05T13:20:38.662589+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:38.662589+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14696448, 'num_objects': 3588, 'num_object_clones': 0, 'num_object_copies': 7176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14503, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -799}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:40.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:40.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17122", 'reported_seq': 17268, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:35.056818+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:35.056818+0000', 'last_peered': '2024-09-05T13:20:35.056818+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:35.056818+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15384576, 'num_objects': 3756, 'num_object_clones': 0, 'num_object_copies': 7512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13273, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:40.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:40.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'17291", 'reported_seq': 17484, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:35.205021+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:35.205021+0000', 'last_peered': '2024-09-05T13:20:35.205021+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:35.205021+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15618048, 'num_objects': 3813, 'num_object_clones': 0, 'num_object_copies': 7626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13190, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -631}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:40.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:40.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17317", 'reported_seq': 17460, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:35.183019+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:35.183019+0000', 'last_peered': '2024-09-05T13:20:35.183019+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:35.183019+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15413248, 'num_objects': 3763, 'num_object_clones': 0, 'num_object_copies': 7526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13447, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:40.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:40.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'17435", 'reported_seq': 17580, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:35.192113+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:35.192113+0000', 'last_peered': '2024-09-05T13:20:35.192113+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:35.192113+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15389155, 'num_objects': 3758, 'num_object_clones': 0, 'num_object_copies': 7516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3758, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14103, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:40.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:41.252 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:41.252 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43619/119034 objects degraded (36.644%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":568,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59517,"data_bytes":243765774,"bytes_used":1865207808,"bytes_avail":288045084672,"bytes_total":289910292480,"degraded_objects":43619,"degraded_total":119034,"degraded_ratio":0.36644152090999216,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":249,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":466},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:41.286 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:41.286 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:41.857 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:41.882 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-05T13:20:41.701815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":238367246,"num_objects":58199,"num_object_clones":0,"num_object_copies":116398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232992,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9880,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818932,"kb_used_data":684084,"kb_used_omap":12,"kb_used_meta":1134771,"kb_avail":281296588,"statfs":{"total":289910292480,"available":288047706112,"internally_reserved":0,"allocated":700502016,"data_stored":699626507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1162005510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,7,5,5,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24084480,"num_objects":-5880,"num_object_clones":0,"num_object_copies":-11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5880,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004475"},"pg_stats":[{"pgid":"2.f","version":"47'17321","reported_seq":17401,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.269044+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:40.269044+0000","last_peered":"2024-09-05T13:20:40.269044+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:40.269044+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14954496,"num_objects":3651,"num_object_clones":0,"num_object_copies":7302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17321,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-770}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17151","reported_seq":17261,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.575741+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:38.575741+0000","last_peered":"2024-09-05T13:20:38.575741+0000","last_clean":"2024-09-05T13:20:38.575741+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:38.575741+0000","last_undegraded":"2024-09-05T13:20:38.575741+0000","last_fullsized":"2024-09-05T13:20:38.575741+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14757888,"num_objects":3603,"num_object_clones":0,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15161,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17465","reported_seq":19698,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.703549+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:38.703549+0000","last_peered":"2024-09-05T13:20:38.703549+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:38.703549+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278104,"num_objects":3731,"num_object_clones":0,"num_object_copies":7462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15276,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1049,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17390","reported_seq":17465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.660914+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:38.660914+0000","last_peered":"2024-09-05T13:20:38.660914+0000","last_clean":"2024-09-05T13:20:38.660914+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:38.660914+0000","last_undegraded":"2024-09-05T13:20:38.660914+0000","last_fullsized":"2024-09-05T13:20:38.660914+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14893056,"num_objects":3636,"num_object_clones":0,"num_object_copies":7272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17390,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17323","reported_seq":17463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:40.244953+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:40.244953+0000","last_peered":"2024-09-05T13:20:40.244953+0000","last_clean":"2024-09-05T13:20:40.244953+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:40.244953+0000","last_undegraded":"2024-09-05T13:20:40.244953+0000","last_fullsized":"2024-09-05T13:20:40.244953+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14581760,"num_objects":3560,"num_object_clones":0,"num_object_copies":7120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13863,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17464","reported_seq":17608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.151326+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:40.151326+0000","last_peered":"2024-09-05T13:20:40.151326+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:40.151326+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15118336,"num_objects":3691,"num_object_clones":0,"num_object_copies":7382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13919,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17104","reported_seq":26136,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:39.100584+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:39.100584+0000","last_peered":"2024-09-05T13:20:39.100584+0000","last_clean":"2024-09-05T13:20:39.100584+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:39.100584+0000","last_undegraded":"2024-09-05T13:20:39.100584+0000","last_fullsized":"2024-09-05T13:20:39.100584+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434304,"num_objects":3524,"num_object_clones":0,"num_object_copies":7048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13423,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17474","reported_seq":17611,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:40.203588+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:40.203588+0000","last_peered":"2024-09-05T13:20:40.203588+0000","last_clean":"2024-09-05T13:20:40.203588+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:40.203588+0000","last_undegraded":"2024-09-05T13:20:40.203588+0000","last_fullsized":"2024-09-05T13:20:40.203588+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14868480,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14084,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17347","reported_seq":17487,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.527575+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:38.527575+0000","last_peered":"2024-09-05T13:20:38.527575+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:38.527575+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15343616,"num_objects":3746,"num_object_clones":0,"num_object_copies":7492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15001,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17271","reported_seq":17415,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.170665+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:40.170665+0000","last_peered":"2024-09-05T13:20:40.170665+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:40.170665+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14905344,"num_objects":3639,"num_object_clones":0,"num_object_copies":7278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17201","reported_seq":26108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.704990+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:38.704990+0000","last_peered":"2024-09-05T13:20:38.704990+0000","last_clean":"2024-09-05T13:20:38.704990+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:38.704990+0000","last_undegraded":"2024-09-05T13:20:38.704990+0000","last_fullsized":"2024-09-05T13:20:38.704990+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15196160,"num_objects":3710,"num_object_clones":0,"num_object_copies":7420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14796,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16994","reported_seq":17123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.662589+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:38.662589+0000","last_peered":"2024-09-05T13:20:38.662589+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:38.662589+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14503,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17277","reported_seq":17422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.265896+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:40.265896+0000","last_peered":"2024-09-05T13:20:40.265896+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:40.265896+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13428,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17450","reported_seq":17643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.191766+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:40.191766+0000","last_peered":"2024-09-05T13:20:40.191766+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:40.191766+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14966784,"num_objects":3654,"num_object_clones":0,"num_object_copies":7308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13349,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-790}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17449","reported_seq":17594,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.106053+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:40.106053+0000","last_peered":"2024-09-05T13:20:40.106053+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:40.106053+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14872576,"num_objects":3631,"num_object_clones":0,"num_object_copies":7262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13579,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17591","reported_seq":17737,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.205520+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:40.205520+0000","last_peered":"2024-09-05T13:20:40.205520+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:40.205520+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14750179,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14259,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-963}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":238367227,"num_objects":58197,"num_object_clones":0,"num_object_copies":116394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232990,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9879,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696295424,"data_stored":696215020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594184,"kb_used_data":261236,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93777656,"statfs":{"total":96636764160,"available":96028319744,"internally_reserved":0,"allocated":267505664,"data_stored":267235662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":340913837},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682348,"kb_used_data":245340,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93689492,"statfs":{"total":96636764160,"available":95938039808,"internally_reserved":0,"allocated":251228160,"data_stored":250958153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542400,"kb_used_data":177508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93829440,"statfs":{"total":96636764160,"available":96081346560,"internally_reserved":0,"allocated":181768192,"data_stored":181432692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":249933824,"data_stored":249930211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":179978240,"data_stored":179909134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266383360,"data_stored":266375675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:44.884 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:45.431 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:45.432 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:45.467 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-05T13:20:43.702461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":238367246,"num_objects":58199,"num_object_clones":0,"num_object_copies":116398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232992,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9880,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818932,"kb_used_data":684084,"kb_used_omap":12,"kb_used_meta":1134771,"kb_avail":281296588,"statfs":{"total":289910292480,"available":288047706112,"internally_reserved":0,"allocated":700502016,"data_stored":699626507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1162005510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,7,5,5,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18784256,"num_objects":-4586,"num_object_clones":0,"num_object_copies":-9172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4586,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004292"},"pg_stats":[{"pgid":"2.f","version":"47'17321","reported_seq":17401,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.269044+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:40.269044+0000","last_peered":"2024-09-05T13:20:40.269044+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:40.269044+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14954496,"num_objects":3651,"num_object_clones":0,"num_object_copies":7302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17321,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-770}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17151","reported_seq":17261,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.575741+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:38.575741+0000","last_peered":"2024-09-05T13:20:38.575741+0000","last_clean":"2024-09-05T13:20:38.575741+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:38.575741+0000","last_undegraded":"2024-09-05T13:20:38.575741+0000","last_fullsized":"2024-09-05T13:20:38.575741+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14757888,"num_objects":3603,"num_object_clones":0,"num_object_copies":7206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15161,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17465","reported_seq":19698,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.703549+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:38.703549+0000","last_peered":"2024-09-05T13:20:38.703549+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:38.703549+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15278104,"num_objects":3731,"num_object_clones":0,"num_object_copies":7462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15276,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1049,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17390","reported_seq":17465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.660914+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:38.660914+0000","last_peered":"2024-09-05T13:20:38.660914+0000","last_clean":"2024-09-05T13:20:38.660914+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:38.660914+0000","last_undegraded":"2024-09-05T13:20:38.660914+0000","last_fullsized":"2024-09-05T13:20:38.660914+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14893056,"num_objects":3636,"num_object_clones":0,"num_object_copies":7272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17390,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17323","reported_seq":17463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:40.244953+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:40.244953+0000","last_peered":"2024-09-05T13:20:40.244953+0000","last_clean":"2024-09-05T13:20:40.244953+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:40.244953+0000","last_undegraded":"2024-09-05T13:20:40.244953+0000","last_fullsized":"2024-09-05T13:20:40.244953+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14581760,"num_objects":3560,"num_object_clones":0,"num_object_copies":7120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13863,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17464","reported_seq":17608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.151326+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:40.151326+0000","last_peered":"2024-09-05T13:20:40.151326+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:40.151326+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15118336,"num_objects":3691,"num_object_clones":0,"num_object_copies":7382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13919,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17104","reported_seq":26136,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:39.100584+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:39.100584+0000","last_peered":"2024-09-05T13:20:39.100584+0000","last_clean":"2024-09-05T13:20:39.100584+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:39.100584+0000","last_undegraded":"2024-09-05T13:20:39.100584+0000","last_fullsized":"2024-09-05T13:20:39.100584+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434304,"num_objects":3524,"num_object_clones":0,"num_object_copies":7048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13423,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17474","reported_seq":17611,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:40.203588+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:40.203588+0000","last_peered":"2024-09-05T13:20:40.203588+0000","last_clean":"2024-09-05T13:20:40.203588+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:40.203588+0000","last_undegraded":"2024-09-05T13:20:40.203588+0000","last_fullsized":"2024-09-05T13:20:40.203588+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14868480,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14084,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17347","reported_seq":17487,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.527575+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:38.527575+0000","last_peered":"2024-09-05T13:20:38.527575+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:38.527575+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15343616,"num_objects":3746,"num_object_clones":0,"num_object_copies":7492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15001,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17271","reported_seq":17415,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.170665+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:40.170665+0000","last_peered":"2024-09-05T13:20:40.170665+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:40.170665+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14905344,"num_objects":3639,"num_object_clones":0,"num_object_copies":7278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17201","reported_seq":26108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.704990+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:38.704990+0000","last_peered":"2024-09-05T13:20:38.704990+0000","last_clean":"2024-09-05T13:20:38.704990+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:38.704990+0000","last_undegraded":"2024-09-05T13:20:38.704990+0000","last_fullsized":"2024-09-05T13:20:38.704990+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15196160,"num_objects":3710,"num_object_clones":0,"num_object_copies":7420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14796,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16994","reported_seq":17123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:38.662589+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:38.662589+0000","last_peered":"2024-09-05T13:20:38.662589+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:38.662589+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14503,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17277","reported_seq":17422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.265896+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:40.265896+0000","last_peered":"2024-09-05T13:20:40.265896+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:40.265896+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13428,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17450","reported_seq":17643,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.191766+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:40.191766+0000","last_peered":"2024-09-05T13:20:40.191766+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:40.191766+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14966784,"num_objects":3654,"num_object_clones":0,"num_object_copies":7308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13349,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-790}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17449","reported_seq":17594,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.106053+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:40.106053+0000","last_peered":"2024-09-05T13:20:40.106053+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:40.106053+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14872576,"num_objects":3631,"num_object_clones":0,"num_object_copies":7262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13579,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17591","reported_seq":17737,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:40.205520+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:40.205520+0000","last_peered":"2024-09-05T13:20:40.205520+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:40.205520+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14750179,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14259,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-963}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":238367227,"num_objects":58197,"num_object_clones":0,"num_object_copies":116394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232990,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9879,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696295424,"data_stored":696215020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594184,"kb_used_data":261236,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93777656,"statfs":{"total":96636764160,"available":96028319744,"internally_reserved":0,"allocated":267505664,"data_stored":267235662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":340913837},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682348,"kb_used_data":245340,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93689492,"statfs":{"total":96636764160,"available":95938039808,"internally_reserved":0,"allocated":251228160,"data_stored":250958153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542400,"kb_used_data":177508,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93829440,"statfs":{"total":96636764160,"available":96081346560,"internally_reserved":0,"allocated":181768192,"data_stored":181432692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":249933824,"data_stored":249930211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":179978240,"data_stored":179909134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266383360,"data_stored":266375675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:45.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:46.019 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:46.022 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:46.045 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-05T13:20:45.703259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":228856334,"num_objects":55877,"num_object_clones":0,"num_object_copies":111754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55877,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235314,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9967,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160697,"ondisk_log_size":160697,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1803052,"kb_used_data":665404,"kb_used_omap":12,"kb_used_meta":1137587,"kb_avail":281312468,"statfs":{"total":289910292480,"available":288063967232,"internally_reserved":0,"allocated":681373696,"data_stored":680498187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1164889094},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,6,8,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28295168,"num_objects":-6908,"num_object_clones":0,"num_object_copies":-13816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6908,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004295"},"pg_stats":[{"pgid":"2.f","version":"47'17454","reported_seq":17532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.275695+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:45.275695+0000","last_peered":"2024-09-05T13:20:45.275695+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:45.275695+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14409728,"num_objects":3518,"num_object_clones":0,"num_object_copies":7036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17454,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-903}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17303","reported_seq":17413,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:43.711907+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:43.711907+0000","last_peered":"2024-09-05T13:20:43.711907+0000","last_clean":"2024-09-05T13:20:43.711907+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:43.711907+0000","last_undegraded":"2024-09-05T13:20:43.711907+0000","last_fullsized":"2024-09-05T13:20:43.711907+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135296,"num_objects":3451,"num_object_clones":0,"num_object_copies":6902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15313,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17605","reported_seq":20011,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:43.694430+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:43.694430+0000","last_peered":"2024-09-05T13:20:43.694430+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:43.694430+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14704664,"num_objects":3591,"num_object_clones":0,"num_object_copies":7182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3591,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15416,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1136,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":234}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17515","reported_seq":17590,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:43.667746+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:43.667746+0000","last_peered":"2024-09-05T13:20:43.667746+0000","last_clean":"2024-09-05T13:20:43.667746+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:43.667746+0000","last_undegraded":"2024-09-05T13:20:43.667746+0000","last_fullsized":"2024-09-05T13:20:43.667746+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14381056,"num_objects":3511,"num_object_clones":0,"num_object_copies":7022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17515,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17475","reported_seq":17615,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:45.213230+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:45.213230+0000","last_peered":"2024-09-05T13:20:45.213230+0000","last_clean":"2024-09-05T13:20:45.213230+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:45.213230+0000","last_undegraded":"2024-09-05T13:20:45.213230+0000","last_fullsized":"2024-09-05T13:20:45.213230+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14015,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17631","reported_seq":17775,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.195860+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:45.195860+0000","last_peered":"2024-09-05T13:20:45.195860+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:45.195860+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434304,"num_objects":3524,"num_object_clones":0,"num_object_copies":7048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14086,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17241","reported_seq":26273,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:44.161618+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:44.161618+0000","last_peered":"2024-09-05T13:20:44.161618+0000","last_clean":"2024-09-05T13:20:44.161618+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:44.161618+0000","last_undegraded":"2024-09-05T13:20:44.161618+0000","last_fullsized":"2024-09-05T13:20:44.161618+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13873152,"num_objects":3387,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13560,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17605","reported_seq":17742,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:45.186935+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:45.186935+0000","last_peered":"2024-09-05T13:20:45.186935+0000","last_clean":"2024-09-05T13:20:45.186935+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:45.186935+0000","last_undegraded":"2024-09-05T13:20:45.186935+0000","last_fullsized":"2024-09-05T13:20:45.186935+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14331904,"num_objects":3499,"num_object_clones":0,"num_object_copies":6998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17482","reported_seq":17622,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:43.639941+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:43.639941+0000","last_peered":"2024-09-05T13:20:43.639941+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:43.639941+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14790656,"num_objects":3611,"num_object_clones":0,"num_object_copies":7222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15136,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17415","reported_seq":17559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.225746+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:45.225746+0000","last_peered":"2024-09-05T13:20:45.225746+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:45.225746+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13782,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-959}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17371","reported_seq":26278,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:43.698946+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:43.698946+0000","last_peered":"2024-09-05T13:20:43.698946+0000","last_clean":"2024-09-05T13:20:43.698946+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:43.698946+0000","last_undegraded":"2024-09-05T13:20:43.698946+0000","last_fullsized":"2024-09-05T13:20:43.698946+0000","mapping_epoch":44,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14499840,"num_objects":3540,"num_object_clones":0,"num_object_copies":7080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14966,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17142","reported_seq":17271,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:43.681731+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:43.681731+0000","last_peered":"2024-09-05T13:20:43.681731+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:43.681731+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14090240,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14651,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17427","reported_seq":17570,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.173733+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:45.173733+0000","last_peered":"2024-09-05T13:20:45.173733+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:45.173733+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135296,"num_objects":3451,"num_object_clones":0,"num_object_copies":6902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13578,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17583","reported_seq":17776,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.273783+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:45.273783+0000","last_peered":"2024-09-05T13:20:45.273783+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:45.273783+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14422016,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13482,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17587","reported_seq":17732,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.183202+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:45.183202+0000","last_peered":"2024-09-05T13:20:45.183202+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:45.183202+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13717,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17758","reported_seq":17904,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.214373+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:45.214373+0000","last_peered":"2024-09-05T13:20:45.214373+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:45.214373+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14066147,"num_objects":3435,"num_object_clones":0,"num_object_copies":6870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14426,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":228856315,"num_objects":55875,"num_object_clones":0,"num_object_copies":111750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55875,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235312,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9966,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":677154816,"data_stored":677074412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160694,"ondisk_log_size":160694,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577048,"kb_used_data":256584,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93794792,"statfs":{"total":96636764160,"available":96045867008,"internally_reserved":0,"allocated":262742016,"data_stored":262472014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":328134317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685176,"kb_used_data":238440,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93686664,"statfs":{"total":96636764160,"available":95935143936,"internally_reserved":0,"allocated":244162560,"data_stored":243892553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540828,"kb_used_data":170380,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93831012,"statfs":{"total":96636764160,"available":96082956288,"internally_reserved":0,"allocated":174469120,"data_stored":174133620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379317935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242757632,"data_stored":242754019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":172724224,"data_stored":172655118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":261672960,"data_stored":261665275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:46.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17454", 'reported_seq': 17532, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.275695+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:45.275695+0000', 'last_peered': '2024-09-05T13:20:45.275695+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:45.275695+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14409728, 'num_objects': 3518, 'num_object_clones': 0, 'num_object_copies': 7036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17454, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -903}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17605", 'reported_seq': 20011, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:43.694430+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:43.694430+0000', 'last_peered': '2024-09-05T13:20:43.694430+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:43.694430+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14704664, 'num_objects': 3591, 'num_object_clones': 0, 'num_object_copies': 7182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3357, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3591, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15416, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1136, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 234}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'17631", 'reported_seq': 17775, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.195860+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:45.195860+0000', 'last_peered': '2024-09-05T13:20:45.195860+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:45.195860+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14434304, 'num_objects': 3524, 'num_object_clones': 0, 'num_object_copies': 7048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14086, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17482", 'reported_seq': 17622, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:43.639941+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:43.639941+0000', 'last_peered': '2024-09-05T13:20:43.639941+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:43.639941+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14790656, 'num_objects': 3611, 'num_object_clones': 0, 'num_object_copies': 7222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3611, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15136, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17415", 'reported_seq': 17559, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.225746+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:45.225746+0000', 'last_peered': '2024-09-05T13:20:45.225746+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:45.225746+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14315520, 'num_objects': 3495, 'num_object_clones': 0, 'num_object_copies': 6990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13782, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -959}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:46.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'17142", 'reported_seq': 17271, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:43.681731+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:43.681731+0000', 'last_peered': '2024-09-05T13:20:43.681731+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:43.681731+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14090240, 'num_objects': 3440, 'num_object_clones': 0, 'num_object_copies': 6880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14651, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -947}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17427", 'reported_seq': 17570, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.173733+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:45.173733+0000', 'last_peered': '2024-09-05T13:20:45.173733+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:45.173733+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14135296, 'num_objects': 3451, 'num_object_clones': 0, 'num_object_copies': 6902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13578, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -968}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'17583", 'reported_seq': 17776, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.273783+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:45.273783+0000', 'last_peered': '2024-09-05T13:20:45.273783+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:45.273783+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14422016, 'num_objects': 3521, 'num_object_clones': 0, 'num_object_copies': 7042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13482, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -923}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17587", 'reported_seq': 17732, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.183202+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:45.183202+0000', 'last_peered': '2024-09-05T13:20:45.183202+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:45.183202+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14307328, 'num_objects': 3493, 'num_object_clones': 0, 'num_object_copies': 6986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13717, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:46.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'17758", 'reported_seq': 17904, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.214373+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:45.214373+0000', 'last_peered': '2024-09-05T13:20:45.214373+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:45.214373+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14066147, 'num_objects': 3435, 'num_object_clones': 0, 'num_object_copies': 6870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14426, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:46.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:46.725 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:46.725 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43532/111754 objects degraded (38.953%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":573,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":55877,"data_bytes":228856334,"bytes_used":1846325248,"bytes_avail":288063967232,"bytes_total":289910292480,"degraded_objects":43532,"degraded_total":111754,"degraded_ratio":0.38953415537698871,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":256,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":575},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:46.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:46.758 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:47.305 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:47.330 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-05T13:20:45.703259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":228856334,"num_objects":55877,"num_object_clones":0,"num_object_copies":111754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55877,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235314,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9967,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160697,"ondisk_log_size":160697,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1803052,"kb_used_data":665404,"kb_used_omap":12,"kb_used_meta":1137587,"kb_avail":281312468,"statfs":{"total":289910292480,"available":288063967232,"internally_reserved":0,"allocated":681373696,"data_stored":680498187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1164889094},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,5,6,8,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28295168,"num_objects":-6908,"num_object_clones":0,"num_object_copies":-13816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6908,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004295"},"pg_stats":[{"pgid":"2.f","version":"47'17454","reported_seq":17532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.275695+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:45.275695+0000","last_peered":"2024-09-05T13:20:45.275695+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:45.275695+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14409728,"num_objects":3518,"num_object_clones":0,"num_object_copies":7036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17454,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-903}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17303","reported_seq":17413,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:43.711907+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:43.711907+0000","last_peered":"2024-09-05T13:20:43.711907+0000","last_clean":"2024-09-05T13:20:43.711907+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:43.711907+0000","last_undegraded":"2024-09-05T13:20:43.711907+0000","last_fullsized":"2024-09-05T13:20:43.711907+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135296,"num_objects":3451,"num_object_clones":0,"num_object_copies":6902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15313,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17605","reported_seq":20011,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:43.694430+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:43.694430+0000","last_peered":"2024-09-05T13:20:43.694430+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:43.694430+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14704664,"num_objects":3591,"num_object_clones":0,"num_object_copies":7182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3357,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3591,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15416,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1136,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":234}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17515","reported_seq":17590,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:43.667746+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:43.667746+0000","last_peered":"2024-09-05T13:20:43.667746+0000","last_clean":"2024-09-05T13:20:43.667746+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:43.667746+0000","last_undegraded":"2024-09-05T13:20:43.667746+0000","last_fullsized":"2024-09-05T13:20:43.667746+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14381056,"num_objects":3511,"num_object_clones":0,"num_object_copies":7022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17515,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17475","reported_seq":17615,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:45.213230+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:45.213230+0000","last_peered":"2024-09-05T13:20:45.213230+0000","last_clean":"2024-09-05T13:20:45.213230+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:45.213230+0000","last_undegraded":"2024-09-05T13:20:45.213230+0000","last_fullsized":"2024-09-05T13:20:45.213230+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14015,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17631","reported_seq":17775,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.195860+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:45.195860+0000","last_peered":"2024-09-05T13:20:45.195860+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:45.195860+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434304,"num_objects":3524,"num_object_clones":0,"num_object_copies":7048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14086,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17241","reported_seq":26273,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:44.161618+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:44.161618+0000","last_peered":"2024-09-05T13:20:44.161618+0000","last_clean":"2024-09-05T13:20:44.161618+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:44.161618+0000","last_undegraded":"2024-09-05T13:20:44.161618+0000","last_fullsized":"2024-09-05T13:20:44.161618+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13873152,"num_objects":3387,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13560,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17605","reported_seq":17742,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:45.186935+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:45.186935+0000","last_peered":"2024-09-05T13:20:45.186935+0000","last_clean":"2024-09-05T13:20:45.186935+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:45.186935+0000","last_undegraded":"2024-09-05T13:20:45.186935+0000","last_fullsized":"2024-09-05T13:20:45.186935+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14331904,"num_objects":3499,"num_object_clones":0,"num_object_copies":6998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17482","reported_seq":17622,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:43.639941+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:43.639941+0000","last_peered":"2024-09-05T13:20:43.639941+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:43.639941+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14790656,"num_objects":3611,"num_object_clones":0,"num_object_copies":7222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15136,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17415","reported_seq":17559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.225746+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:45.225746+0000","last_peered":"2024-09-05T13:20:45.225746+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:45.225746+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13782,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-959}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17371","reported_seq":26278,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:43.698946+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:43.698946+0000","last_peered":"2024-09-05T13:20:43.698946+0000","last_clean":"2024-09-05T13:20:43.698946+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:43.698946+0000","last_undegraded":"2024-09-05T13:20:43.698946+0000","last_fullsized":"2024-09-05T13:20:43.698946+0000","mapping_epoch":44,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14499840,"num_objects":3540,"num_object_clones":0,"num_object_copies":7080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14966,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17142","reported_seq":17271,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:43.681731+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:43.681731+0000","last_peered":"2024-09-05T13:20:43.681731+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:43.681731+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14090240,"num_objects":3440,"num_object_clones":0,"num_object_copies":6880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14651,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17427","reported_seq":17570,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.173733+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:45.173733+0000","last_peered":"2024-09-05T13:20:45.173733+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:45.173733+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135296,"num_objects":3451,"num_object_clones":0,"num_object_copies":6902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13578,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17583","reported_seq":17776,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.273783+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:45.273783+0000","last_peered":"2024-09-05T13:20:45.273783+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:45.273783+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14422016,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13482,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17587","reported_seq":17732,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.183202+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:45.183202+0000","last_peered":"2024-09-05T13:20:45.183202+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:45.183202+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13717,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17758","reported_seq":17904,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.214373+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:45.214373+0000","last_peered":"2024-09-05T13:20:45.214373+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:45.214373+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14066147,"num_objects":3435,"num_object_clones":0,"num_object_copies":6870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14426,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":228856315,"num_objects":55875,"num_object_clones":0,"num_object_copies":111750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55875,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":235312,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9966,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":677154816,"data_stored":677074412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160694,"ondisk_log_size":160694,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577048,"kb_used_data":256584,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93794792,"statfs":{"total":96636764160,"available":96045867008,"internally_reserved":0,"allocated":262742016,"data_stored":262472014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":328134317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685176,"kb_used_data":238440,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93686664,"statfs":{"total":96636764160,"available":95935143936,"internally_reserved":0,"allocated":244162560,"data_stored":243892553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540828,"kb_used_data":170380,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93831012,"statfs":{"total":96636764160,"available":96082956288,"internally_reserved":0,"allocated":174469120,"data_stored":174133620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379317935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242757632,"data_stored":242754019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":172724224,"data_stored":172655118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":261672960,"data_stored":261665275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:50.332 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:50.864 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:50.866 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:50.899 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-05T13:20:49.704546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224723470,"num_objects":54868,"num_object_clones":0,"num_object_copies":109736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54868,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236323,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10051,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804188,"kb_used_data":654060,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281311332,"statfs":{"total":289910292480,"available":288062803968,"internally_reserved":0,"allocated":669757440,"data_stored":668881931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1177668614},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,6,5,12,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22986752,"num_objects":-5612,"num_object_clones":0,"num_object_copies":-11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5612,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004108"},"pg_stats":[{"pgid":"2.f","version":"47'17454","reported_seq":17532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.275695+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:45.275695+0000","last_peered":"2024-09-05T13:20:45.275695+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:45.275695+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14409728,"num_objects":3518,"num_object_clones":0,"num_object_copies":7036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17454,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-903}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17462","reported_seq":17572,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.735175+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:48.735175+0000","last_peered":"2024-09-05T13:20:48.735175+0000","last_clean":"2024-09-05T13:20:48.735175+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:48.735175+0000","last_undegraded":"2024-09-05T13:20:48.735175+0000","last_fullsized":"2024-09-05T13:20:48.735175+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13484032,"num_objects":3292,"num_object_clones":0,"num_object_copies":6584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17747","reported_seq":20322,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.704593+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:48.704593+0000","last_peered":"2024-09-05T13:20:48.704593+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:48.704593+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14123032,"num_objects":3449,"num_object_clones":0,"num_object_copies":6898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15558,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":176}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17670","reported_seq":17745,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.731256+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:48.731256+0000","last_peered":"2024-09-05T13:20:48.731256+0000","last_clean":"2024-09-05T13:20:48.731256+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:48.731256+0000","last_undegraded":"2024-09-05T13:20:48.731256+0000","last_fullsized":"2024-09-05T13:20:48.731256+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13746176,"num_objects":3356,"num_object_clones":0,"num_object_copies":6712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17670,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17475","reported_seq":17615,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:45.213230+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:45.213230+0000","last_peered":"2024-09-05T13:20:45.213230+0000","last_clean":"2024-09-05T13:20:45.213230+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:45.213230+0000","last_undegraded":"2024-09-05T13:20:45.213230+0000","last_fullsized":"2024-09-05T13:20:45.213230+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14015,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17631","reported_seq":17775,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.195860+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:45.195860+0000","last_peered":"2024-09-05T13:20:45.195860+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:45.195860+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434304,"num_objects":3524,"num_object_clones":0,"num_object_copies":7048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14086,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17376","reported_seq":26408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:49.337421+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:49.337421+0000","last_peered":"2024-09-05T13:20:49.337421+0000","last_clean":"2024-09-05T13:20:49.337421+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:49.337421+0000","last_undegraded":"2024-09-05T13:20:49.337421+0000","last_fullsized":"2024-09-05T13:20:49.337421+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13320192,"num_objects":3252,"num_object_clones":0,"num_object_copies":6504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17605","reported_seq":17742,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:45.186935+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:45.186935+0000","last_peered":"2024-09-05T13:20:45.186935+0000","last_clean":"2024-09-05T13:20:45.186935+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:45.186935+0000","last_undegraded":"2024-09-05T13:20:45.186935+0000","last_fullsized":"2024-09-05T13:20:45.186935+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14331904,"num_objects":3499,"num_object_clones":0,"num_object_copies":6998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17621","reported_seq":17762,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.733650+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:48.733650+0000","last_peered":"2024-09-05T13:20:48.733650+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:48.733650+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221312,"num_objects":3472,"num_object_clones":0,"num_object_copies":6944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15275,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-979}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17415","reported_seq":17559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.225746+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:45.225746+0000","last_peered":"2024-09-05T13:20:45.225746+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:45.225746+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13782,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-959}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17507","reported_seq":26414,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.660169+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:48.660169+0000","last_peered":"2024-09-05T13:20:48.660169+0000","last_clean":"2024-09-05T13:20:48.660169+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:48.660169+0000","last_undegraded":"2024-09-05T13:20:48.660169+0000","last_fullsized":"2024-09-05T13:20:48.660169+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13942784,"num_objects":3404,"num_object_clones":0,"num_object_copies":6808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15102,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17285","reported_seq":17414,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.659059+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:48.659059+0000","last_peered":"2024-09-05T13:20:48.659059+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:48.659059+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14794,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1090}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17427","reported_seq":17570,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.173733+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:45.173733+0000","last_peered":"2024-09-05T13:20:45.173733+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:45.173733+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135296,"num_objects":3451,"num_object_clones":0,"num_object_copies":6902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13578,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17583","reported_seq":17776,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.273783+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:45.273783+0000","last_peered":"2024-09-05T13:20:45.273783+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:45.273783+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14422016,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13482,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17587","reported_seq":17732,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.183202+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:45.183202+0000","last_peered":"2024-09-05T13:20:45.183202+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:45.183202+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13717,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17758","reported_seq":17904,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.214373+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:45.214373+0000","last_peered":"2024-09-05T13:20:45.214373+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:45.214373+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14066147,"num_objects":3435,"num_object_clones":0,"num_object_copies":6870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14426,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224723451,"num_objects":54866,"num_object_clones":0,"num_object_copies":109732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236321,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10050,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":665309184,"data_stored":665228780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576696,"kb_used_data":252072,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93795144,"statfs":{"total":96636764160,"available":96046227456,"internally_reserved":0,"allocated":258121728,"data_stored":257851726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685176,"kb_used_data":238440,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93686664,"statfs":{"total":96636764160,"available":95935143936,"internally_reserved":0,"allocated":244162560,"data_stored":243892553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542316,"kb_used_data":163548,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93829524,"statfs":{"total":96636764160,"available":96081432576,"internally_reserved":0,"allocated":167473152,"data_stored":167137652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242757632,"data_stored":242754019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165548032,"data_stored":165478926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257003520,"data_stored":256995835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:50.901 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:51.462 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:51.463 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:51.488 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-05T13:20:49.704546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224723470,"num_objects":54868,"num_object_clones":0,"num_object_copies":109736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54868,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236323,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10051,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804188,"kb_used_data":654060,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281311332,"statfs":{"total":289910292480,"available":288062803968,"internally_reserved":0,"allocated":669757440,"data_stored":668881931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1177668614},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,6,5,12,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22986752,"num_objects":-5612,"num_object_clones":0,"num_object_copies":-11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5612,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004108"},"pg_stats":[{"pgid":"2.f","version":"47'17454","reported_seq":17532,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.275695+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:45.275695+0000","last_peered":"2024-09-05T13:20:45.275695+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:45.275695+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14409728,"num_objects":3518,"num_object_clones":0,"num_object_copies":7036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17454,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-903}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17462","reported_seq":17572,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.735175+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:48.735175+0000","last_peered":"2024-09-05T13:20:48.735175+0000","last_clean":"2024-09-05T13:20:48.735175+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:48.735175+0000","last_undegraded":"2024-09-05T13:20:48.735175+0000","last_fullsized":"2024-09-05T13:20:48.735175+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13484032,"num_objects":3292,"num_object_clones":0,"num_object_copies":6584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17747","reported_seq":20322,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.704593+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:48.704593+0000","last_peered":"2024-09-05T13:20:48.704593+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:48.704593+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14123032,"num_objects":3449,"num_object_clones":0,"num_object_copies":6898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15558,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":176}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17670","reported_seq":17745,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.731256+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:48.731256+0000","last_peered":"2024-09-05T13:20:48.731256+0000","last_clean":"2024-09-05T13:20:48.731256+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:48.731256+0000","last_undegraded":"2024-09-05T13:20:48.731256+0000","last_fullsized":"2024-09-05T13:20:48.731256+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13746176,"num_objects":3356,"num_object_clones":0,"num_object_copies":6712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17670,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17475","reported_seq":17615,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:45.213230+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:45.213230+0000","last_peered":"2024-09-05T13:20:45.213230+0000","last_clean":"2024-09-05T13:20:45.213230+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:45.213230+0000","last_undegraded":"2024-09-05T13:20:45.213230+0000","last_fullsized":"2024-09-05T13:20:45.213230+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14015,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17631","reported_seq":17775,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.195860+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:45.195860+0000","last_peered":"2024-09-05T13:20:45.195860+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:45.195860+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434304,"num_objects":3524,"num_object_clones":0,"num_object_copies":7048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14086,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17376","reported_seq":26408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:49.337421+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:49.337421+0000","last_peered":"2024-09-05T13:20:49.337421+0000","last_clean":"2024-09-05T13:20:49.337421+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:49.337421+0000","last_undegraded":"2024-09-05T13:20:49.337421+0000","last_fullsized":"2024-09-05T13:20:49.337421+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13320192,"num_objects":3252,"num_object_clones":0,"num_object_copies":6504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17605","reported_seq":17742,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:45.186935+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:45.186935+0000","last_peered":"2024-09-05T13:20:45.186935+0000","last_clean":"2024-09-05T13:20:45.186935+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:45.186935+0000","last_undegraded":"2024-09-05T13:20:45.186935+0000","last_fullsized":"2024-09-05T13:20:45.186935+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14331904,"num_objects":3499,"num_object_clones":0,"num_object_copies":6998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14215,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17621","reported_seq":17762,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.733650+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:48.733650+0000","last_peered":"2024-09-05T13:20:48.733650+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:48.733650+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221312,"num_objects":3472,"num_object_clones":0,"num_object_copies":6944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15275,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-979}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17415","reported_seq":17559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.225746+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:45.225746+0000","last_peered":"2024-09-05T13:20:45.225746+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:45.225746+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13782,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-959}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17507","reported_seq":26414,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.660169+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:48.660169+0000","last_peered":"2024-09-05T13:20:48.660169+0000","last_clean":"2024-09-05T13:20:48.660169+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:48.660169+0000","last_undegraded":"2024-09-05T13:20:48.660169+0000","last_fullsized":"2024-09-05T13:20:48.660169+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13942784,"num_objects":3404,"num_object_clones":0,"num_object_copies":6808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15102,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17285","reported_seq":17414,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.659059+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:48.659059+0000","last_peered":"2024-09-05T13:20:48.659059+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:48.659059+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14794,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1090}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17427","reported_seq":17570,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.173733+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:45.173733+0000","last_peered":"2024-09-05T13:20:45.173733+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:45.173733+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135296,"num_objects":3451,"num_object_clones":0,"num_object_copies":6902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13578,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17583","reported_seq":17776,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.273783+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:45.273783+0000","last_peered":"2024-09-05T13:20:45.273783+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:45.273783+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14422016,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13482,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-923}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17587","reported_seq":17732,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.183202+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:45.183202+0000","last_peered":"2024-09-05T13:20:45.183202+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:45.183202+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13717,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17758","reported_seq":17904,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:45.214373+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:45.214373+0000","last_peered":"2024-09-05T13:20:45.214373+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:45.214373+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14066147,"num_objects":3435,"num_object_clones":0,"num_object_copies":6870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14426,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224723451,"num_objects":54866,"num_object_clones":0,"num_object_copies":109732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236321,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10050,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":665309184,"data_stored":665228780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576696,"kb_used_data":252072,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93795144,"statfs":{"total":96636764160,"available":96046227456,"internally_reserved":0,"allocated":258121728,"data_stored":257851726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685176,"kb_used_data":238440,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93686664,"statfs":{"total":96636764160,"available":95935143936,"internally_reserved":0,"allocated":244162560,"data_stored":243892553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,1,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542316,"kb_used_data":163548,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93829524,"statfs":{"total":96636764160,"available":96081432576,"internally_reserved":0,"allocated":167473152,"data_stored":167137652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242757632,"data_stored":242754019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165548032,"data_stored":165478926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257003520,"data_stored":256995835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17454", 'reported_seq': 17532, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.275695+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:45.275695+0000', 'last_peered': '2024-09-05T13:20:45.275695+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:45.275695+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14409728, 'num_objects': 3518, 'num_object_clones': 0, 'num_object_copies': 7036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17454, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -903}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17747", 'reported_seq': 20322, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:48.704593+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:48.704593+0000', 'last_peered': '2024-09-05T13:20:48.704593+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:48.704593+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14123032, 'num_objects': 3449, 'num_object_clones': 0, 'num_object_copies': 6898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3273, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3449, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15558, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1220, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 176}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'17631", 'reported_seq': 17775, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.195860+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:45.195860+0000', 'last_peered': '2024-09-05T13:20:45.195860+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:45.195860+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14434304, 'num_objects': 3524, 'num_object_clones': 0, 'num_object_copies': 7048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14086, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17621", 'reported_seq': 17762, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:48.733650+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:48.733650+0000', 'last_peered': '2024-09-05T13:20:48.733650+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:48.733650+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14221312, 'num_objects': 3472, 'num_object_clones': 0, 'num_object_copies': 6944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15275, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -979}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:51.490 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17415", 'reported_seq': 17559, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.225746+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:45.225746+0000', 'last_peered': '2024-09-05T13:20:45.225746+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:45.225746+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14315520, 'num_objects': 3495, 'num_object_clones': 0, 'num_object_copies': 6990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13782, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -959}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'17285", 'reported_seq': 17414, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:48.659059+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:48.659059+0000', 'last_peered': '2024-09-05T13:20:48.659059+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:48.659059+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13504512, 'num_objects': 3297, 'num_object_clones': 0, 'num_object_copies': 6594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3297, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14794, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1090}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17427", 'reported_seq': 17570, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.173733+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:45.173733+0000', 'last_peered': '2024-09-05T13:20:45.173733+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:45.173733+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14135296, 'num_objects': 3451, 'num_object_clones': 0, 'num_object_copies': 6902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13578, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -968}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'17583", 'reported_seq': 17776, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.273783+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:45.273783+0000', 'last_peered': '2024-09-05T13:20:45.273783+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:45.273783+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14422016, 'num_objects': 3521, 'num_object_clones': 0, 'num_object_copies': 7042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13482, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -923}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17587", 'reported_seq': 17732, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.183202+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:45.183202+0000', 'last_peered': '2024-09-05T13:20:45.183202+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:45.183202+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14307328, 'num_objects': 3493, 'num_object_clones': 0, 'num_object_copies': 6986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13717, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:51.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'17758", 'reported_seq': 17904, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:45.214373+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:45.214373+0000', 'last_peered': '2024-09-05T13:20:45.214373+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:45.214373+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14066147, 'num_objects': 3435, 'num_object_clones': 0, 'num_object_copies': 6870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14426, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:51.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:52.172 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:52.172 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43448/107110 objects degraded (40.564%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":579,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":53555,"data_bytes":219345422,"bytes_used":1844776960,"bytes_avail":288065515520,"bytes_total":289910292480,"degraded_objects":43448,"degraded_total":107110,"degraded_ratio":0.40563906264587807,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":496},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:52.197 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:52.197 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:52.758 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:52.760 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:52.783 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-05T13:20:51.705375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":219345422,"num_objects":53555,"num_object_clones":0,"num_object_copies":107110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53555,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":237636,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10051,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1801540,"kb_used_data":647244,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281313980,"statfs":{"total":289910292480,"available":288065515520,"internally_reserved":0,"allocated":662777856,"data_stored":661902347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1181928454},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,6,11,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24420352,"num_objects":-5962,"num_object_clones":0,"num_object_copies":-11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5962,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004194"},"pg_stats":[{"pgid":"2.f","version":"47'17594","reported_seq":17673,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:50.304160+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:50.304160+0000","last_peered":"2024-09-05T13:20:50.304160+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:50.304160+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13836288,"num_objects":3378,"num_object_clones":0,"num_object_copies":6756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17594,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17462","reported_seq":17572,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.735175+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:48.735175+0000","last_peered":"2024-09-05T13:20:48.735175+0000","last_clean":"2024-09-05T13:20:48.735175+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:48.735175+0000","last_undegraded":"2024-09-05T13:20:48.735175+0000","last_fullsized":"2024-09-05T13:20:48.735175+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13484032,"num_objects":3292,"num_object_clones":0,"num_object_copies":6584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17747","reported_seq":20322,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.704593+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:48.704593+0000","last_peered":"2024-09-05T13:20:48.704593+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:48.704593+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14123032,"num_objects":3449,"num_object_clones":0,"num_object_copies":6898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15558,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":176}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17670","reported_seq":17745,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.731256+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:48.731256+0000","last_peered":"2024-09-05T13:20:48.731256+0000","last_clean":"2024-09-05T13:20:48.731256+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:48.731256+0000","last_undegraded":"2024-09-05T13:20:48.731256+0000","last_fullsized":"2024-09-05T13:20:48.731256+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13746176,"num_objects":3356,"num_object_clones":0,"num_object_copies":6712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17670,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17623","reported_seq":17763,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:50.145561+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:50.145561+0000","last_peered":"2024-09-05T13:20:50.145561+0000","last_clean":"2024-09-05T13:20:50.145561+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:50.145561+0000","last_undegraded":"2024-09-05T13:20:50.145561+0000","last_fullsized":"2024-09-05T13:20:50.145561+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13352960,"num_objects":3260,"num_object_clones":0,"num_object_copies":6520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14163,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17762","reported_seq":17906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:50.253730+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:50.253730+0000","last_peered":"2024-09-05T13:20:50.253730+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:50.253730+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13897728,"num_objects":3393,"num_object_clones":0,"num_object_copies":6786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14217,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1058}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17376","reported_seq":26408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:49.337421+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:49.337421+0000","last_peered":"2024-09-05T13:20:49.337421+0000","last_clean":"2024-09-05T13:20:49.337421+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:49.337421+0000","last_undegraded":"2024-09-05T13:20:49.337421+0000","last_fullsized":"2024-09-05T13:20:49.337421+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13320192,"num_objects":3252,"num_object_clones":0,"num_object_copies":6504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17732","reported_seq":17870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:50.207659+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:50.207659+0000","last_peered":"2024-09-05T13:20:50.207659+0000","last_clean":"2024-09-05T13:20:50.207659+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:50.207659+0000","last_undegraded":"2024-09-05T13:20:50.207659+0000","last_fullsized":"2024-09-05T13:20:50.207659+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13811712,"num_objects":3372,"num_object_clones":0,"num_object_copies":6744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14342,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17621","reported_seq":17762,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.733650+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:48.733650+0000","last_peered":"2024-09-05T13:20:48.733650+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:48.733650+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221312,"num_objects":3472,"num_object_clones":0,"num_object_copies":6944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15275,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-979}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17555","reported_seq":17699,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:50.237680+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:50.237680+0000","last_peered":"2024-09-05T13:20:50.237680+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:50.237680+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13742080,"num_objects":3355,"num_object_clones":0,"num_object_copies":6710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13922,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17507","reported_seq":26414,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:48.660169+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:48.660169+0000","last_peered":"2024-09-05T13:20:48.660169+0000","last_clean":"2024-09-05T13:20:48.660169+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:48.660169+0000","last_undegraded":"2024-09-05T13:20:48.660169+0000","last_fullsized":"2024-09-05T13:20:48.660169+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13942784,"num_objects":3404,"num_object_clones":0,"num_object_copies":6808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15102,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17285","reported_seq":17414,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:48.659059+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:48.659059+0000","last_peered":"2024-09-05T13:20:48.659059+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:48.659059+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14794,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1090}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17560","reported_seq":17706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:50.254330+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:50.254330+0000","last_peered":"2024-09-05T13:20:50.254330+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:50.254330+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13590528,"num_objects":3318,"num_object_clones":0,"num_object_copies":6636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13711,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17757","reported_seq":17950,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:50.303725+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:50.303725+0000","last_peered":"2024-09-05T13:20:50.303725+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:50.303725+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13709312,"num_objects":3347,"num_object_clones":0,"num_object_copies":6694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13656,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1097}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17761","reported_seq":17905,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:50.277818+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:50.277818+0000","last_peered":"2024-09-05T13:20:50.277818+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:50.277818+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13594624,"num_objects":3319,"num_object_clones":0,"num_object_copies":6638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13891,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17904","reported_seq":18050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:50.284100+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:50.284100+0000","last_peered":"2024-09-05T13:20:50.284100+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:50.284100+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13468131,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1276}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":219345403,"num_objects":53553,"num_object_clones":0,"num_object_copies":107106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":237634,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10050,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":658169856,"data_stored":658089452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576696,"kb_used_data":252072,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93795144,"statfs":{"total":96636764160,"available":96046227456,"internally_reserved":0,"allocated":258121728,"data_stored":257851726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682528,"kb_used_data":231624,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93689312,"statfs":{"total":96636764160,"available":95937855488,"internally_reserved":0,"allocated":237182976,"data_stored":236912969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542316,"kb_used_data":163548,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93829524,"statfs":{"total":96636764160,"available":96081432576,"internally_reserved":0,"allocated":167473152,"data_stored":167137652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":235618304,"data_stored":235614691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165548032,"data_stored":165478926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257003520,"data_stored":256995835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:55.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:56.334 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:56.334 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:56.356 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-05T13:20:55.709821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209949198,"num_objects":51261,"num_object_clones":0,"num_object_copies":102522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51261,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239930,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10127,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160613,"ondisk_log_size":160613,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769056,"kb_used_data":627244,"kb_used_omap":12,"kb_used_meta":1141747,"kb_avail":281346464,"statfs":{"total":289910292480,"available":288098779136,"internally_reserved":0,"allocated":642297856,"data_stored":641422347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1169148934},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,8,10,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28418048,"num_objects":-6938,"num_object_clones":0,"num_object_copies":-13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6938,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007359"},"pg_stats":[{"pgid":"2.f","version":"47'17730","reported_seq":17810,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.271932+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:55.271932+0000","last_peered":"2024-09-05T13:20:55.271932+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:55.271932+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13279232,"num_objects":3242,"num_object_clones":0,"num_object_copies":6484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17730,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17600","reported_seq":17710,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.721732+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:53.721732+0000","last_peered":"2024-09-05T13:20:53.721732+0000","last_clean":"2024-09-05T13:20:53.721732+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:53.721732+0000","last_undegraded":"2024-09-05T13:20:53.721732+0000","last_fullsized":"2024-09-05T13:20:53.721732+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12918784,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15610,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17901","reported_seq":20628,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.755244+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:53.755244+0000","last_peered":"2024-09-05T13:20:53.755244+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:53.755244+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13492248,"num_objects":3295,"num_object_clones":0,"num_object_copies":6590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3295,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15712,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":98}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17793","reported_seq":17868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.625055+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:53.625055+0000","last_peered":"2024-09-05T13:20:53.625055+0000","last_clean":"2024-09-05T13:20:53.625055+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:53.625055+0000","last_undegraded":"2024-09-05T13:20:53.625055+0000","last_fullsized":"2024-09-05T13:20:53.625055+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17793,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17744","reported_seq":17884,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:55.337213+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:55.337213+0000","last_peered":"2024-09-05T13:20:55.337213+0000","last_clean":"2024-09-05T13:20:55.337213+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:55.337213+0000","last_undegraded":"2024-09-05T13:20:55.337213+0000","last_fullsized":"2024-09-05T13:20:55.337213+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12857344,"num_objects":3139,"num_object_clones":0,"num_object_copies":6278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14284,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17906","reported_seq":18050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.313020+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:55.313020+0000","last_peered":"2024-09-05T13:20:55.313020+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:55.313020+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13307904,"num_objects":3249,"num_object_clones":0,"num_object_copies":6498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1202}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17508","reported_seq":26540,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:54.367229+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:54.367229+0000","last_peered":"2024-09-05T13:20:54.367229+0000","last_clean":"2024-09-05T13:20:54.367229+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:54.367229+0000","last_undegraded":"2024-09-05T13:20:54.367229+0000","last_fullsized":"2024-09-05T13:20:54.367229+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12779520,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17889","reported_seq":18027,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:55.312477+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:55.312477+0000","last_peered":"2024-09-05T13:20:55.312477+0000","last_clean":"2024-09-05T13:20:55.312477+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:55.312477+0000","last_undegraded":"2024-09-05T13:20:55.312477+0000","last_fullsized":"2024-09-05T13:20:55.312477+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13168640,"num_objects":3215,"num_object_clones":0,"num_object_copies":6430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14499,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17758","reported_seq":17899,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.723638+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:53.723638+0000","last_peered":"2024-09-05T13:20:53.723638+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:53.723638+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13660160,"num_objects":3335,"num_object_clones":0,"num_object_copies":6670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15412,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1116}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17711","reported_seq":17853,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.322673+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:55.322673+0000","last_peered":"2024-09-05T13:20:55.322673+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:55.322673+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13103104,"num_objects":3199,"num_object_clones":0,"num_object_copies":6398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14078,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17660","reported_seq":26567,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.678818+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:53.678818+0000","last_peered":"2024-09-05T13:20:53.678818+0000","last_clean":"2024-09-05T13:20:53.678818+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:53.678818+0000","last_undegraded":"2024-09-05T13:20:53.678818+0000","last_fullsized":"2024-09-05T13:20:53.678818+0000","mapping_epoch":44,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13316096,"num_objects":3251,"num_object_clones":0,"num_object_copies":6502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17416","reported_seq":17545,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.661063+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:53.661063+0000","last_peered":"2024-09-05T13:20:53.661063+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:53.661063+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12967936,"num_objects":3166,"num_object_clones":0,"num_object_copies":6332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14925,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17720","reported_seq":17865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.348111+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:55.348111+0000","last_peered":"2024-09-05T13:20:55.348111+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:55.348111+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12935168,"num_objects":3158,"num_object_clones":0,"num_object_copies":6316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13871,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17908","reported_seq":18101,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.383910+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:55.383910+0000","last_peered":"2024-09-05T13:20:55.383910+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:55.383910+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17917","reported_seq":18061,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.343977+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:55.343977+0000","last_peered":"2024-09-05T13:20:55.343977+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:55.343977+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12955648,"num_objects":3163,"num_object_clones":0,"num_object_copies":6326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14047,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18049","reported_seq":18195,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.244591+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:55.244591+0000","last_peered":"2024-09-05T13:20:55.244591+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:55.244591+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874211,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":209949179,"num_objects":51259,"num_object_clones":0,"num_object_copies":102518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239928,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10126,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":639328256,"data_stored":639247852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576280,"kb_used_data":247496,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93795560,"statfs":{"total":96636764160,"available":96046653440,"internally_reserved":0,"allocated":253435904,"data_stored":253165902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667252,"kb_used_data":224676,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93704588,"statfs":{"total":96636764160,"available":95953498112,"internally_reserved":0,"allocated":230068224,"data_stored":229798217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525524,"kb_used_data":155072,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93846316,"statfs":{"total":96636764160,"available":96098627584,"internally_reserved":0,"allocated":158793728,"data_stored":158458228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379317935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,8,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228552704,"data_stored":228549091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":158359552,"data_stored":158290446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252416000,"data_stored":252408315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:56.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:56.951 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:56.979 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-05T13:20:55.709821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209949198,"num_objects":51261,"num_object_clones":0,"num_object_copies":102522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51261,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239930,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10127,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160613,"ondisk_log_size":160613,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769056,"kb_used_data":627244,"kb_used_omap":12,"kb_used_meta":1141747,"kb_avail":281346464,"statfs":{"total":289910292480,"available":288098779136,"internally_reserved":0,"allocated":642297856,"data_stored":641422347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1169148934},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,8,10,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28418048,"num_objects":-6938,"num_object_clones":0,"num_object_copies":-13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6938,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007359"},"pg_stats":[{"pgid":"2.f","version":"47'17730","reported_seq":17810,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.271932+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:55.271932+0000","last_peered":"2024-09-05T13:20:55.271932+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:55.271932+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13279232,"num_objects":3242,"num_object_clones":0,"num_object_copies":6484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17730,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17600","reported_seq":17710,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.721732+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:53.721732+0000","last_peered":"2024-09-05T13:20:53.721732+0000","last_clean":"2024-09-05T13:20:53.721732+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:53.721732+0000","last_undegraded":"2024-09-05T13:20:53.721732+0000","last_fullsized":"2024-09-05T13:20:53.721732+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12918784,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15610,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17901","reported_seq":20628,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.755244+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:53.755244+0000","last_peered":"2024-09-05T13:20:53.755244+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:53.755244+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13492248,"num_objects":3295,"num_object_clones":0,"num_object_copies":6590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3295,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15712,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":98}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17793","reported_seq":17868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.625055+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:53.625055+0000","last_peered":"2024-09-05T13:20:53.625055+0000","last_clean":"2024-09-05T13:20:53.625055+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:53.625055+0000","last_undegraded":"2024-09-05T13:20:53.625055+0000","last_fullsized":"2024-09-05T13:20:53.625055+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17793,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17744","reported_seq":17884,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:55.337213+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:55.337213+0000","last_peered":"2024-09-05T13:20:55.337213+0000","last_clean":"2024-09-05T13:20:55.337213+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:55.337213+0000","last_undegraded":"2024-09-05T13:20:55.337213+0000","last_fullsized":"2024-09-05T13:20:55.337213+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12857344,"num_objects":3139,"num_object_clones":0,"num_object_copies":6278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14284,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17906","reported_seq":18050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.313020+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:55.313020+0000","last_peered":"2024-09-05T13:20:55.313020+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:55.313020+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13307904,"num_objects":3249,"num_object_clones":0,"num_object_copies":6498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1202}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17508","reported_seq":26540,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:54.367229+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:54.367229+0000","last_peered":"2024-09-05T13:20:54.367229+0000","last_clean":"2024-09-05T13:20:54.367229+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:54.367229+0000","last_undegraded":"2024-09-05T13:20:54.367229+0000","last_fullsized":"2024-09-05T13:20:54.367229+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12779520,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17889","reported_seq":18027,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:55.312477+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:55.312477+0000","last_peered":"2024-09-05T13:20:55.312477+0000","last_clean":"2024-09-05T13:20:55.312477+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:55.312477+0000","last_undegraded":"2024-09-05T13:20:55.312477+0000","last_fullsized":"2024-09-05T13:20:55.312477+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13168640,"num_objects":3215,"num_object_clones":0,"num_object_copies":6430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14499,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17758","reported_seq":17899,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.723638+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:53.723638+0000","last_peered":"2024-09-05T13:20:53.723638+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:53.723638+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13660160,"num_objects":3335,"num_object_clones":0,"num_object_copies":6670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15412,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1116}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17711","reported_seq":17853,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.322673+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:55.322673+0000","last_peered":"2024-09-05T13:20:55.322673+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:55.322673+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13103104,"num_objects":3199,"num_object_clones":0,"num_object_copies":6398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14078,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17660","reported_seq":26567,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.678818+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:53.678818+0000","last_peered":"2024-09-05T13:20:53.678818+0000","last_clean":"2024-09-05T13:20:53.678818+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:53.678818+0000","last_undegraded":"2024-09-05T13:20:53.678818+0000","last_fullsized":"2024-09-05T13:20:53.678818+0000","mapping_epoch":44,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13316096,"num_objects":3251,"num_object_clones":0,"num_object_copies":6502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17416","reported_seq":17545,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.661063+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:53.661063+0000","last_peered":"2024-09-05T13:20:53.661063+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:53.661063+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12967936,"num_objects":3166,"num_object_clones":0,"num_object_copies":6332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14925,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17720","reported_seq":17865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.348111+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:55.348111+0000","last_peered":"2024-09-05T13:20:55.348111+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:55.348111+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12935168,"num_objects":3158,"num_object_clones":0,"num_object_copies":6316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13871,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17908","reported_seq":18101,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.383910+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:55.383910+0000","last_peered":"2024-09-05T13:20:55.383910+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:55.383910+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17917","reported_seq":18061,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.343977+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:55.343977+0000","last_peered":"2024-09-05T13:20:55.343977+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:55.343977+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12955648,"num_objects":3163,"num_object_clones":0,"num_object_copies":6326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14047,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18049","reported_seq":18195,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.244591+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:55.244591+0000","last_peered":"2024-09-05T13:20:55.244591+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:55.244591+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874211,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":209949179,"num_objects":51259,"num_object_clones":0,"num_object_copies":102518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239928,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10126,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":639328256,"data_stored":639247852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576280,"kb_used_data":247496,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93795560,"statfs":{"total":96636764160,"available":96046653440,"internally_reserved":0,"allocated":253435904,"data_stored":253165902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667252,"kb_used_data":224676,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93704588,"statfs":{"total":96636764160,"available":95953498112,"internally_reserved":0,"allocated":230068224,"data_stored":229798217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525524,"kb_used_data":155072,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93846316,"statfs":{"total":96636764160,"available":96098627584,"internally_reserved":0,"allocated":158793728,"data_stored":158458228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379317935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,8,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228552704,"data_stored":228549091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":158359552,"data_stored":158290446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252416000,"data_stored":252408315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:20:56.980 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:20:56.980 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17730", 'reported_seq': 17810, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:55.271932+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:20:55.271932+0000', 'last_peered': '2024-09-05T13:20:55.271932+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:20:55.271932+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13279232, 'num_objects': 3242, 'num_object_clones': 0, 'num_object_copies': 6484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3242, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17730, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1179}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:56.980 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'17901", 'reported_seq': 20628, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:53.755244+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:53.755244+0000', 'last_peered': '2024-09-05T13:20:53.755244+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:53.755244+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13492248, 'num_objects': 3295, 'num_object_clones': 0, 'num_object_copies': 6590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3197, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3295, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15712, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1296, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 98}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'17906", 'reported_seq': 18050, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:55.313020+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:20:55.313020+0000', 'last_peered': '2024-09-05T13:20:55.313020+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:20:55.313020+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13307904, 'num_objects': 3249, 'num_object_clones': 0, 'num_object_copies': 6498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14361, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1202}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17758", 'reported_seq': 17899, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:53.723638+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:53.723638+0000', 'last_peered': '2024-09-05T13:20:53.723638+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:53.723638+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13660160, 'num_objects': 3335, 'num_object_clones': 0, 'num_object_copies': 6670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15412, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1116}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17711", 'reported_seq': 17853, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:55.322673+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:20:55.322673+0000', 'last_peered': '2024-09-05T13:20:55.322673+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:20:55.322673+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13103104, 'num_objects': 3199, 'num_object_clones': 0, 'num_object_copies': 6398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14078, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1255}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'17416", 'reported_seq': 17545, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:53.661063+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:53.661063+0000', 'last_peered': '2024-09-05T13:20:53.661063+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:53.661063+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12967936, 'num_objects': 3166, 'num_object_clones': 0, 'num_object_copies': 6332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3166, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14925, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1221}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:20:56.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17720", 'reported_seq': 17865, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:55.348111+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:20:55.348111+0000', 'last_peered': '2024-09-05T13:20:55.348111+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:20:55.348111+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12935168, 'num_objects': 3158, 'num_object_clones': 0, 'num_object_copies': 6316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13871, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1261}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:56.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:20:56.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'17908", 'reported_seq': 18101, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:55.383910+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:20:55.383910+0000', 'last_peered': '2024-09-05T13:20:55.383910+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:20:55.383910+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13090816, 'num_objects': 3196, 'num_object_clones': 0, 'num_object_copies': 6392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3196, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13807, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:56.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:20:56.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'17917", 'reported_seq': 18061, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:55.343977+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:20:55.343977+0000', 'last_peered': '2024-09-05T13:20:55.343977+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:20:55.343977+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12955648, 'num_objects': 3163, 'num_object_clones': 0, 'num_object_copies': 6326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3163, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14047, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1403}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:56.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:20:56.982 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'18049", 'reported_seq': 18195, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:55.244591+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:20:55.244591+0000', 'last_peered': '2024-09-05T13:20:55.244591+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:20:55.244591+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12874211, 'num_objects': 3144, 'num_object_clones': 0, 'num_object_copies': 6288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3144, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14717, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:20:56.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:20:57.665 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:57.665 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43372/102522 objects degraded (42.305%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":584,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":51261,"data_bytes":209949198,"bytes_used":1811513344,"bytes_avail":288098779136,"bytes_total":289910292480,"degraded_objects":43372,"degraded_total":102522,"degraded_ratio":0.42305066229687294,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":577},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:20:57.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:20:57.695 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:20:58.262 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:20:58.264 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:20:58.292 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-05T13:20:57.710475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":209949198,"num_objects":51261,"num_object_clones":0,"num_object_copies":102522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51261,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239930,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10127,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160613,"ondisk_log_size":160613,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769056,"kb_used_data":627244,"kb_used_omap":12,"kb_used_meta":1141747,"kb_avail":281346464,"statfs":{"total":289910292480,"available":288098779136,"internally_reserved":0,"allocated":642297856,"data_stored":641422347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1169148934},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,8,10,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18907136,"num_objects":-4616,"num_object_clones":0,"num_object_copies":-9232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4616,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007215"},"pg_stats":[{"pgid":"2.f","version":"47'17730","reported_seq":17810,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.271932+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:20:55.271932+0000","last_peered":"2024-09-05T13:20:55.271932+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:20:55.271932+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13279232,"num_objects":3242,"num_object_clones":0,"num_object_copies":6484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17730,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1179}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17600","reported_seq":17710,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.721732+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:53.721732+0000","last_peered":"2024-09-05T13:20:53.721732+0000","last_clean":"2024-09-05T13:20:53.721732+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:53.721732+0000","last_undegraded":"2024-09-05T13:20:53.721732+0000","last_fullsized":"2024-09-05T13:20:53.721732+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12918784,"num_objects":3154,"num_object_clones":0,"num_object_copies":6308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15610,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17901","reported_seq":20628,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.755244+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:53.755244+0000","last_peered":"2024-09-05T13:20:53.755244+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:53.755244+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13492248,"num_objects":3295,"num_object_clones":0,"num_object_copies":6590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3295,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15712,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1296,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":98}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17793","reported_seq":17868,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.625055+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:53.625055+0000","last_peered":"2024-09-05T13:20:53.625055+0000","last_clean":"2024-09-05T13:20:53.625055+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:53.625055+0000","last_undegraded":"2024-09-05T13:20:53.625055+0000","last_fullsized":"2024-09-05T13:20:53.625055+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17793,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17744","reported_seq":17884,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:55.337213+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:20:55.337213+0000","last_peered":"2024-09-05T13:20:55.337213+0000","last_clean":"2024-09-05T13:20:55.337213+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:20:55.337213+0000","last_undegraded":"2024-09-05T13:20:55.337213+0000","last_fullsized":"2024-09-05T13:20:55.337213+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12857344,"num_objects":3139,"num_object_clones":0,"num_object_copies":6278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14284,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17906","reported_seq":18050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.313020+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:20:55.313020+0000","last_peered":"2024-09-05T13:20:55.313020+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:20:55.313020+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13307904,"num_objects":3249,"num_object_clones":0,"num_object_copies":6498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1202}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17508","reported_seq":26540,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:54.367229+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:54.367229+0000","last_peered":"2024-09-05T13:20:54.367229+0000","last_clean":"2024-09-05T13:20:54.367229+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:54.367229+0000","last_undegraded":"2024-09-05T13:20:54.367229+0000","last_fullsized":"2024-09-05T13:20:54.367229+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12779520,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'17889","reported_seq":18027,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:55.312477+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:20:55.312477+0000","last_peered":"2024-09-05T13:20:55.312477+0000","last_clean":"2024-09-05T13:20:55.312477+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:20:55.312477+0000","last_undegraded":"2024-09-05T13:20:55.312477+0000","last_fullsized":"2024-09-05T13:20:55.312477+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13168640,"num_objects":3215,"num_object_clones":0,"num_object_copies":6430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14499,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17758","reported_seq":17899,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.723638+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:53.723638+0000","last_peered":"2024-09-05T13:20:53.723638+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:53.723638+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13660160,"num_objects":3335,"num_object_clones":0,"num_object_copies":6670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15412,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1116}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17711","reported_seq":17853,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.322673+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:20:55.322673+0000","last_peered":"2024-09-05T13:20:55.322673+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:20:55.322673+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13103104,"num_objects":3199,"num_object_clones":0,"num_object_copies":6398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14078,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1255}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17660","reported_seq":26567,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:53.678818+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:53.678818+0000","last_peered":"2024-09-05T13:20:53.678818+0000","last_clean":"2024-09-05T13:20:53.678818+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:53.678818+0000","last_undegraded":"2024-09-05T13:20:53.678818+0000","last_fullsized":"2024-09-05T13:20:53.678818+0000","mapping_epoch":44,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13316096,"num_objects":3251,"num_object_clones":0,"num_object_copies":6502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17416","reported_seq":17545,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:53.661063+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:53.661063+0000","last_peered":"2024-09-05T13:20:53.661063+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:53.661063+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12967936,"num_objects":3166,"num_object_clones":0,"num_object_copies":6332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14925,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17720","reported_seq":17865,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.348111+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:20:55.348111+0000","last_peered":"2024-09-05T13:20:55.348111+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:20:55.348111+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12935168,"num_objects":3158,"num_object_clones":0,"num_object_copies":6316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13871,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17908","reported_seq":18101,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.383910+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:20:55.383910+0000","last_peered":"2024-09-05T13:20:55.383910+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:20:55.383910+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13807,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17917","reported_seq":18061,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.343977+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:20:55.343977+0000","last_peered":"2024-09-05T13:20:55.343977+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:20:55.343977+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12955648,"num_objects":3163,"num_object_clones":0,"num_object_copies":6326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14047,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18049","reported_seq":18195,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:55.244591+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:20:55.244591+0000","last_peered":"2024-09-05T13:20:55.244591+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:20:55.244591+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12874211,"num_objects":3144,"num_object_clones":0,"num_object_copies":6288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":209949179,"num_objects":51259,"num_object_clones":0,"num_object_copies":102518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239928,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10126,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":639328256,"data_stored":639247852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576280,"kb_used_data":247496,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93795560,"statfs":{"total":96636764160,"available":96046653440,"internally_reserved":0,"allocated":253435904,"data_stored":253165902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667252,"kb_used_data":224676,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93704588,"statfs":{"total":96636764160,"available":95953498112,"internally_reserved":0,"allocated":230068224,"data_stored":229798217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525524,"kb_used_data":155072,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93846316,"statfs":{"total":96636764160,"available":96098627584,"internally_reserved":0,"allocated":158793728,"data_stored":158458228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379317935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,8,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228552704,"data_stored":228549091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":158359552,"data_stored":158290446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":252416000,"data_stored":252408315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:01.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:01.890 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:01.914 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-05T13:21:01.711349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":200638990,"num_objects":48988,"num_object_clones":0,"num_object_copies":97976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242203,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10213,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1764660,"kb_used_data":610364,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281350860,"statfs":{"total":289910292480,"available":288103280640,"internally_reserved":0,"allocated":625012736,"data_stored":624137227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1181928454},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,2,5,5,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24084480,"num_objects":-5880,"num_object_clones":0,"num_object_copies":-11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5880,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006803"},"pg_stats":[{"pgid":"2.f","version":"47'17868","reported_seq":17948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.300449+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:00.300449+0000","last_peered":"2024-09-05T13:21:00.300449+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:00.300449+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12713984,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17868,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17733","reported_seq":17843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.734352+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:58.734352+0000","last_peered":"2024-09-05T13:20:58.734352+0000","last_clean":"2024-09-05T13:20:58.734352+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:58.734352+0000","last_undegraded":"2024-09-05T13:20:58.734352+0000","last_fullsized":"2024-09-05T13:20:58.734352+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12374016,"num_objects":3021,"num_object_clones":0,"num_object_copies":6042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15743,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18056","reported_seq":20955,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.757476+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:58.757476+0000","last_peered":"2024-09-05T13:20:58.757476+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:58.757476+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12857368,"num_objects":3140,"num_object_clones":0,"num_object_copies":6280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15867,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1382,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17924","reported_seq":17999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.729064+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:58.729064+0000","last_peered":"2024-09-05T13:20:58.729064+0000","last_clean":"2024-09-05T13:20:58.729064+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:58.729064+0000","last_undegraded":"2024-09-05T13:20:58.729064+0000","last_fullsized":"2024-09-05T13:20:58.729064+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12705792,"num_objects":3102,"num_object_clones":0,"num_object_copies":6204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17924,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17889","reported_seq":18029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:00.300404+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:00.300404+0000","last_peered":"2024-09-05T13:21:00.300404+0000","last_clean":"2024-09-05T13:21:00.300404+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:00.300404+0000","last_undegraded":"2024-09-05T13:21:00.300404+0000","last_fullsized":"2024-09-05T13:21:00.300404+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12263424,"num_objects":2994,"num_object_clones":0,"num_object_copies":5988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14429,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18078","reported_seq":18220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.357493+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:00.357493+0000","last_peered":"2024-09-05T13:21:00.357493+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:00.357493+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12603392,"num_objects":3077,"num_object_clones":0,"num_object_copies":6154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14533,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17639","reported_seq":26671,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:59.287592+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:59.287592+0000","last_peered":"2024-09-05T13:20:59.287592+0000","last_clean":"2024-09-05T13:20:59.287592+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:59.287592+0000","last_undegraded":"2024-09-05T13:20:59.287592+0000","last_fullsized":"2024-09-05T13:20:59.287592+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12242944,"num_objects":2989,"num_object_clones":0,"num_object_copies":5978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18035","reported_seq":18173,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:00.365706+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:00.365706+0000","last_peered":"2024-09-05T13:21:00.365706+0000","last_clean":"2024-09-05T13:21:00.365706+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:00.365706+0000","last_undegraded":"2024-09-05T13:21:00.365706+0000","last_fullsized":"2024-09-05T13:21:00.365706+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14645,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17897","reported_seq":18038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.706874+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:58.706874+0000","last_peered":"2024-09-05T13:20:58.706874+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:58.706874+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15551,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1255}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17865","reported_seq":18009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.286346+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:00.286346+0000","last_peered":"2024-09-05T13:21:00.286346+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:00.286346+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12472320,"num_objects":3045,"num_object_clones":0,"num_object_copies":6090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14232,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17826","reported_seq":26733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.736378+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:58.736378+0000","last_peered":"2024-09-05T13:20:58.736378+0000","last_clean":"2024-09-05T13:20:58.736378+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:58.736378+0000","last_undegraded":"2024-09-05T13:20:58.736378+0000","last_fullsized":"2024-09-05T13:20:58.736378+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12636160,"num_objects":3085,"num_object_clones":0,"num_object_copies":6170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15421,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17550","reported_seq":17679,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.715027+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:58.715027+0000","last_peered":"2024-09-05T13:20:58.715027+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:58.715027+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15059,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17857","reported_seq":18003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.345718+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:00.345718+0000","last_peered":"2024-09-05T13:21:00.345718+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:00.345718+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12374016,"num_objects":3021,"num_object_clones":0,"num_object_copies":6042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14008,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1398}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18039","reported_seq":18232,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.386758+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:00.386758+0000","last_peered":"2024-09-05T13:21:00.386758+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:00.386758+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13938,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18048","reported_seq":18193,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.334332+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:00.334332+0000","last_peered":"2024-09-05T13:21:00.334332+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:00.334332+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18179","reported_seq":18325,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.230659+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:00.230659+0000","last_peered":"2024-09-05T13:21:00.230659+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:00.230659+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12341731,"num_objects":3014,"num_object_clones":0,"num_object_copies":6028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14847,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":200638971,"num_objects":48986,"num_object_clones":0,"num_object_copies":97972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48986,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242201,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10212,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":620449792,"data_stored":620369388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160883,"ondisk_log_size":160883,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575988,"kb_used_data":243044,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93795852,"statfs":{"total":96636764160,"available":96046952448,"internally_reserved":0,"allocated":248877056,"data_stored":248607054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":340913837},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664572,"kb_used_data":217828,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93707268,"statfs":{"total":96636764160,"available":95956242432,"internally_reserved":0,"allocated":223055872,"data_stored":222785865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,3,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524100,"kb_used_data":149492,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93847740,"statfs":{"total":96636764160,"available":96100085760,"internally_reserved":0,"allocated":153079808,"data_stored":152744308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221564928,"data_stored":221561315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":151130112,"data_stored":151061006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247754752,"data_stored":247747067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:01.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:02.487 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:02.490 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:02.508 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-05T13:21:01.711349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":200638990,"num_objects":48988,"num_object_clones":0,"num_object_copies":97976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242203,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10213,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1764660,"kb_used_data":610364,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281350860,"statfs":{"total":289910292480,"available":288103280640,"internally_reserved":0,"allocated":625012736,"data_stored":624137227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1181928454},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,2,5,5,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24084480,"num_objects":-5880,"num_object_clones":0,"num_object_copies":-11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5880,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006803"},"pg_stats":[{"pgid":"2.f","version":"47'17868","reported_seq":17948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.300449+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:00.300449+0000","last_peered":"2024-09-05T13:21:00.300449+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:00.300449+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12713984,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17868,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17733","reported_seq":17843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.734352+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:58.734352+0000","last_peered":"2024-09-05T13:20:58.734352+0000","last_clean":"2024-09-05T13:20:58.734352+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:58.734352+0000","last_undegraded":"2024-09-05T13:20:58.734352+0000","last_fullsized":"2024-09-05T13:20:58.734352+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12374016,"num_objects":3021,"num_object_clones":0,"num_object_copies":6042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15743,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18056","reported_seq":20955,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.757476+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:58.757476+0000","last_peered":"2024-09-05T13:20:58.757476+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:58.757476+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12857368,"num_objects":3140,"num_object_clones":0,"num_object_copies":6280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15867,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1382,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17924","reported_seq":17999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.729064+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:58.729064+0000","last_peered":"2024-09-05T13:20:58.729064+0000","last_clean":"2024-09-05T13:20:58.729064+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:58.729064+0000","last_undegraded":"2024-09-05T13:20:58.729064+0000","last_fullsized":"2024-09-05T13:20:58.729064+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12705792,"num_objects":3102,"num_object_clones":0,"num_object_copies":6204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17924,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17889","reported_seq":18029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:00.300404+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:00.300404+0000","last_peered":"2024-09-05T13:21:00.300404+0000","last_clean":"2024-09-05T13:21:00.300404+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:00.300404+0000","last_undegraded":"2024-09-05T13:21:00.300404+0000","last_fullsized":"2024-09-05T13:21:00.300404+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12263424,"num_objects":2994,"num_object_clones":0,"num_object_copies":5988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14429,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18078","reported_seq":18220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.357493+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:00.357493+0000","last_peered":"2024-09-05T13:21:00.357493+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:00.357493+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12603392,"num_objects":3077,"num_object_clones":0,"num_object_copies":6154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14533,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17639","reported_seq":26671,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:59.287592+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:59.287592+0000","last_peered":"2024-09-05T13:20:59.287592+0000","last_clean":"2024-09-05T13:20:59.287592+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:59.287592+0000","last_undegraded":"2024-09-05T13:20:59.287592+0000","last_fullsized":"2024-09-05T13:20:59.287592+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12242944,"num_objects":2989,"num_object_clones":0,"num_object_copies":5978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18035","reported_seq":18173,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:00.365706+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:00.365706+0000","last_peered":"2024-09-05T13:21:00.365706+0000","last_clean":"2024-09-05T13:21:00.365706+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:00.365706+0000","last_undegraded":"2024-09-05T13:21:00.365706+0000","last_fullsized":"2024-09-05T13:21:00.365706+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14645,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17897","reported_seq":18038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.706874+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:58.706874+0000","last_peered":"2024-09-05T13:20:58.706874+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:58.706874+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15551,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1255}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17865","reported_seq":18009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.286346+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:00.286346+0000","last_peered":"2024-09-05T13:21:00.286346+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:00.286346+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12472320,"num_objects":3045,"num_object_clones":0,"num_object_copies":6090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14232,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17826","reported_seq":26733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.736378+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:58.736378+0000","last_peered":"2024-09-05T13:20:58.736378+0000","last_clean":"2024-09-05T13:20:58.736378+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:58.736378+0000","last_undegraded":"2024-09-05T13:20:58.736378+0000","last_fullsized":"2024-09-05T13:20:58.736378+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12636160,"num_objects":3085,"num_object_clones":0,"num_object_copies":6170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15421,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17550","reported_seq":17679,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.715027+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:58.715027+0000","last_peered":"2024-09-05T13:20:58.715027+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:58.715027+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15059,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17857","reported_seq":18003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.345718+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:00.345718+0000","last_peered":"2024-09-05T13:21:00.345718+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:00.345718+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12374016,"num_objects":3021,"num_object_clones":0,"num_object_copies":6042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14008,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1398}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18039","reported_seq":18232,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.386758+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:00.386758+0000","last_peered":"2024-09-05T13:21:00.386758+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:00.386758+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13938,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18048","reported_seq":18193,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.334332+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:00.334332+0000","last_peered":"2024-09-05T13:21:00.334332+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:00.334332+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18179","reported_seq":18325,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.230659+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:00.230659+0000","last_peered":"2024-09-05T13:21:00.230659+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:00.230659+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12341731,"num_objects":3014,"num_object_clones":0,"num_object_copies":6028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14847,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":200638971,"num_objects":48986,"num_object_clones":0,"num_object_copies":97972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48986,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242201,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10212,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":620449792,"data_stored":620369388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160883,"ondisk_log_size":160883,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575988,"kb_used_data":243044,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93795852,"statfs":{"total":96636764160,"available":96046952448,"internally_reserved":0,"allocated":248877056,"data_stored":248607054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":340913837},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664572,"kb_used_data":217828,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93707268,"statfs":{"total":96636764160,"available":95956242432,"internally_reserved":0,"allocated":223055872,"data_stored":222785865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,3,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524100,"kb_used_data":149492,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93847740,"statfs":{"total":96636764160,"available":96100085760,"internally_reserved":0,"allocated":153079808,"data_stored":152744308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221564928,"data_stored":221561315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":151130112,"data_stored":151061006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247754752,"data_stored":247747067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:02.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:02.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'17868", 'reported_seq': 17948, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:00.300449+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:00.300449+0000', 'last_peered': '2024-09-05T13:21:00.300449+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:00.300449+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12713984, 'num_objects': 3104, 'num_object_clones': 0, 'num_object_copies': 6208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3104, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17868, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1317}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:02.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:02.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'18056", 'reported_seq': 20955, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:58.757476+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:20:58.757476+0000', 'last_peered': '2024-09-05T13:20:58.757476+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:20:58.757476+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12857368, 'num_objects': 3140, 'num_object_clones': 0, 'num_object_copies': 6280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3140, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15867, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1382, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 29}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:02.509 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'18078", 'reported_seq': 18220, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:00.357493+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:00.357493+0000', 'last_peered': '2024-09-05T13:21:00.357493+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:00.357493+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12603392, 'num_objects': 3077, 'num_object_clones': 0, 'num_object_copies': 6154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14533, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1374}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'17897", 'reported_seq': 18038, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:58.706874+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:20:58.706874+0000', 'last_peered': '2024-09-05T13:20:58.706874+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:20:58.706874+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13090816, 'num_objects': 3196, 'num_object_clones': 0, 'num_object_copies': 6392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3196, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15551, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1255}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17865", 'reported_seq': 18009, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:00.286346+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:00.286346+0000', 'last_peered': '2024-09-05T13:21:00.286346+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:00.286346+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12472320, 'num_objects': 3045, 'num_object_clones': 0, 'num_object_copies': 6090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14232, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1409}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'17550", 'reported_seq': 17679, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:20:58.715027+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:20:58.715027+0000', 'last_peered': '2024-09-05T13:20:58.715027+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:20:58.715027+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12419072, 'num_objects': 3032, 'num_object_clones': 0, 'num_object_copies': 6064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15059, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1355}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17857", 'reported_seq': 18003, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:00.345718+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:00.345718+0000', 'last_peered': '2024-09-05T13:21:00.345718+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:00.345718+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12374016, 'num_objects': 3021, 'num_object_clones': 0, 'num_object_copies': 6042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14008, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1398}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:02.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:02.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'18039", 'reported_seq': 18232, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:00.386758+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:00.386758+0000', 'last_peered': '2024-09-05T13:21:00.386758+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:00.386758+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12554240, 'num_objects': 3065, 'num_object_clones': 0, 'num_object_copies': 6130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13938, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:02.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:02.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'18048", 'reported_seq': 18193, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:00.334332+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:00.334332+0000', 'last_peered': '2024-09-05T13:21:00.334332+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:00.334332+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12419072, 'num_objects': 3032, 'num_object_clones': 0, 'num_object_copies': 6064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14178, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1534}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:02.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:02.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'18179", 'reported_seq': 18325, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:00.230659+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:00.230659+0000', 'last_peered': '2024-09-05T13:21:00.230659+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:00.230659+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12341731, 'num_objects': 3014, 'num_object_clones': 0, 'num_object_copies': 6028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14847, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1551}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:02.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:03.253 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:03.254 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43286/97976 objects degraded (44.180%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":590,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":48988,"data_bytes":200638990,"bytes_used":1807011840,"bytes_avail":288103280640,"bytes_total":289910292480,"degraded_objects":43286,"degraded_total":97976,"degraded_ratio":0.44180207397730059,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":489},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:03.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:03.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:03.865 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:03.868 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:03.887 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-05T13:21:03.711814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":200638990,"num_objects":48988,"num_object_clones":0,"num_object_copies":97976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242203,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10213,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1764660,"kb_used_data":610364,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281350860,"statfs":{"total":289910292480,"available":288103280640,"internally_reserved":0,"allocated":625012736,"data_stored":624137227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1181928454},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,2,5,5,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18706432,"num_objects":-4567,"num_object_clones":0,"num_object_copies":-9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4567,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006439"},"pg_stats":[{"pgid":"2.f","version":"47'17868","reported_seq":17948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.300449+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:00.300449+0000","last_peered":"2024-09-05T13:21:00.300449+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:00.300449+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12713984,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17868,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17733","reported_seq":17843,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.734352+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:20:58.734352+0000","last_peered":"2024-09-05T13:20:58.734352+0000","last_clean":"2024-09-05T13:20:58.734352+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:20:58.734352+0000","last_undegraded":"2024-09-05T13:20:58.734352+0000","last_fullsized":"2024-09-05T13:20:58.734352+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12374016,"num_objects":3021,"num_object_clones":0,"num_object_copies":6042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15743,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18056","reported_seq":20955,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.757476+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:20:58.757476+0000","last_peered":"2024-09-05T13:20:58.757476+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:20:58.757476+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12857368,"num_objects":3140,"num_object_clones":0,"num_object_copies":6280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15867,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1382,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":29}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17924","reported_seq":17999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.729064+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:20:58.729064+0000","last_peered":"2024-09-05T13:20:58.729064+0000","last_clean":"2024-09-05T13:20:58.729064+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:20:58.729064+0000","last_undegraded":"2024-09-05T13:20:58.729064+0000","last_fullsized":"2024-09-05T13:20:58.729064+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12705792,"num_objects":3102,"num_object_clones":0,"num_object_copies":6204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17924,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17889","reported_seq":18029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:00.300404+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:00.300404+0000","last_peered":"2024-09-05T13:21:00.300404+0000","last_clean":"2024-09-05T13:21:00.300404+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:00.300404+0000","last_undegraded":"2024-09-05T13:21:00.300404+0000","last_fullsized":"2024-09-05T13:21:00.300404+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12263424,"num_objects":2994,"num_object_clones":0,"num_object_copies":5988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14429,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18078","reported_seq":18220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.357493+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:00.357493+0000","last_peered":"2024-09-05T13:21:00.357493+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:00.357493+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12603392,"num_objects":3077,"num_object_clones":0,"num_object_copies":6154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14533,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17639","reported_seq":26671,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:59.287592+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:20:59.287592+0000","last_peered":"2024-09-05T13:20:59.287592+0000","last_clean":"2024-09-05T13:20:59.287592+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:20:59.287592+0000","last_undegraded":"2024-09-05T13:20:59.287592+0000","last_fullsized":"2024-09-05T13:20:59.287592+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12242944,"num_objects":2989,"num_object_clones":0,"num_object_copies":5978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18035","reported_seq":18173,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:00.365706+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:00.365706+0000","last_peered":"2024-09-05T13:21:00.365706+0000","last_clean":"2024-09-05T13:21:00.365706+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:00.365706+0000","last_undegraded":"2024-09-05T13:21:00.365706+0000","last_fullsized":"2024-09-05T13:21:00.365706+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14645,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17897","reported_seq":18038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.706874+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:20:58.706874+0000","last_peered":"2024-09-05T13:20:58.706874+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:20:58.706874+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15551,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1255}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17865","reported_seq":18009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.286346+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:00.286346+0000","last_peered":"2024-09-05T13:21:00.286346+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:00.286346+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12472320,"num_objects":3045,"num_object_clones":0,"num_object_copies":6090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14232,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17826","reported_seq":26733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:58.736378+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:20:58.736378+0000","last_peered":"2024-09-05T13:20:58.736378+0000","last_clean":"2024-09-05T13:20:58.736378+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:20:58.736378+0000","last_undegraded":"2024-09-05T13:20:58.736378+0000","last_fullsized":"2024-09-05T13:20:58.736378+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12636160,"num_objects":3085,"num_object_clones":0,"num_object_copies":6170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15421,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17550","reported_seq":17679,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:20:58.715027+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:20:58.715027+0000","last_peered":"2024-09-05T13:20:58.715027+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:20:58.715027+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15059,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17857","reported_seq":18003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.345718+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:00.345718+0000","last_peered":"2024-09-05T13:21:00.345718+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:00.345718+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12374016,"num_objects":3021,"num_object_clones":0,"num_object_copies":6042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14008,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1398}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18039","reported_seq":18232,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.386758+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:00.386758+0000","last_peered":"2024-09-05T13:21:00.386758+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:00.386758+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13938,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18048","reported_seq":18193,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.334332+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:00.334332+0000","last_peered":"2024-09-05T13:21:00.334332+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:00.334332+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1534}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18179","reported_seq":18325,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:00.230659+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:00.230659+0000","last_peered":"2024-09-05T13:21:00.230659+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:00.230659+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12341731,"num_objects":3014,"num_object_clones":0,"num_object_copies":6028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14847,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":200638971,"num_objects":48986,"num_object_clones":0,"num_object_copies":97972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48986,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242201,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10212,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":620449792,"data_stored":620369388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160883,"ondisk_log_size":160883,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575988,"kb_used_data":243044,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93795852,"statfs":{"total":96636764160,"available":96046952448,"internally_reserved":0,"allocated":248877056,"data_stored":248607054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":340913837},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664572,"kb_used_data":217828,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93707268,"statfs":{"total":96636764160,"available":95956242432,"internally_reserved":0,"allocated":223055872,"data_stored":222785865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,3,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524100,"kb_used_data":149492,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93847740,"statfs":{"total":96636764160,"available":96100085760,"internally_reserved":0,"allocated":153079808,"data_stored":152744308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221564928,"data_stored":221561315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":151130112,"data_stored":151061006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":247754752,"data_stored":247747067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:06.889 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:08.230 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:08.232 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:08.256 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-05T13:21:07.712964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":190960142,"num_objects":46625,"num_object_clones":0,"num_object_copies":93250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46625,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244566,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10295,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160949,"ondisk_log_size":160949,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1750096,"kb_used_data":591644,"kb_used_omap":12,"kb_used_meta":1158387,"kb_avail":281365424,"statfs":{"total":289910292480,"available":288118194176,"internally_reserved":0,"allocated":605843456,"data_stored":604967947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1186188294},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,7,0,5,3,13,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18989056,"num_objects":-4636,"num_object_clones":0,"num_object_copies":-9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4636,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003143"},"pg_stats":[{"pgid":"2.f","version":"47'18004","reported_seq":18085,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.362681+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:05.362681+0000","last_peered":"2024-09-05T13:21:05.362681+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:05.362681+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12156928,"num_objects":2968,"num_object_clones":0,"num_object_copies":5936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18004,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17892","reported_seq":18001,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:03.746903+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:03.746903+0000","last_peered":"2024-09-05T13:21:03.746903+0000","last_clean":"2024-09-05T13:21:03.746903+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:03.746903+0000","last_undegraded":"2024-09-05T13:21:03.746903+0000","last_fullsized":"2024-09-05T13:21:03.746903+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15902,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18201","reported_seq":21261,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:03.758945+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:03.758945+0000","last_peered":"2024-09-05T13:21:03.758945+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:03.758945+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12263448,"num_objects":2995,"num_object_clones":0,"num_object_copies":5990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16012,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-34}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18076","reported_seq":18151,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:03.723778+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:03.723778+0000","last_peered":"2024-09-05T13:21:03.723778+0000","last_clean":"2024-09-05T13:21:03.723778+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:03.723778+0000","last_undegraded":"2024-09-05T13:21:03.723778+0000","last_fullsized":"2024-09-05T13:21:03.723778+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12083200,"num_objects":2950,"num_object_clones":0,"num_object_copies":5900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18076,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18027","reported_seq":18167,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:05.288675+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:05.288675+0000","last_peered":"2024-09-05T13:21:05.288675+0000","last_clean":"2024-09-05T13:21:05.288675+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:05.288675+0000","last_undegraded":"2024-09-05T13:21:05.288675+0000","last_fullsized":"2024-09-05T13:21:05.288675+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698176,"num_objects":2856,"num_object_clones":0,"num_object_copies":5712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14567,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18252","reported_seq":18396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.343293+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:05.343293+0000","last_peered":"2024-09-05T13:21:05.343293+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:05.343293+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11890688,"num_objects":2903,"num_object_clones":0,"num_object_copies":5806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14707,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1548}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17787","reported_seq":26819,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:04.350278+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:04.350278+0000","last_peered":"2024-09-05T13:21:04.350278+0000","last_clean":"2024-09-05T13:21:04.350278+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:04.350278+0000","last_undegraded":"2024-09-05T13:21:04.350278+0000","last_fullsized":"2024-09-05T13:21:04.350278+0000","mapping_epoch":46,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11636736,"num_objects":2841,"num_object_clones":0,"num_object_copies":5682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14106,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18181","reported_seq":18319,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:05.362855+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:05.362855+0000","last_peered":"2024-09-05T13:21:05.362855+0000","last_clean":"2024-09-05T13:21:05.362855+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:05.362855+0000","last_undegraded":"2024-09-05T13:21:05.362855+0000","last_fullsized":"2024-09-05T13:21:05.362855+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11972608,"num_objects":2923,"num_object_clones":0,"num_object_copies":5846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18041","reported_seq":18182,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:03.725703+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:03.725703+0000","last_peered":"2024-09-05T13:21:03.725703+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:03.725703+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12500992,"num_objects":3052,"num_object_clones":0,"num_object_copies":6104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15695,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17997","reported_seq":18141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.289224+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:05.289224+0000","last_peered":"2024-09-05T13:21:05.289224+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:05.289224+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11931648,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17982","reported_seq":26889,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:03.684361+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:03.684361+0000","last_peered":"2024-09-05T13:21:03.684361+0000","last_clean":"2024-09-05T13:21:03.684361+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:03.684361+0000","last_undegraded":"2024-09-05T13:21:03.684361+0000","last_fullsized":"2024-09-05T13:21:03.684361+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11997184,"num_objects":2929,"num_object_clones":0,"num_object_copies":5858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15577,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17718","reported_seq":17847,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:03.761613+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:03.761613+0000","last_peered":"2024-09-05T13:21:03.761613+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:03.761613+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11730944,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15227,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1523}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17996","reported_seq":18142,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.417285+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:05.417285+0000","last_peered":"2024-09-05T13:21:05.417285+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:05.417285+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14147,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18181","reported_seq":18374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.413499+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:05.413499+0000","last_peered":"2024-09-05T13:21:05.413499+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:05.413499+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11972608,"num_objects":2923,"num_object_clones":0,"num_object_copies":5846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18196","reported_seq":18341,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.317862+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:05.317862+0000","last_peered":"2024-09-05T13:21:05.317862+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:05.317862+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11812864,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14326,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18315","reported_seq":18461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.339974+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:05.339974+0000","last_peered":"2024-09-05T13:21:05.339974+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:05.339974+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11784675,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":190960123,"num_objects":46623,"num_object_clones":0,"num_object_copies":93246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244564,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10294,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":601419776,"data_stored":601339372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558628,"kb_used_data":238160,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93813212,"statfs":{"total":96636764160,"available":96064729088,"internally_reserved":0,"allocated":243875840,"data_stored":243605838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":328134317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":666120,"kb_used_data":211064,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93705720,"statfs":{"total":96636764160,"available":95954657280,"internally_reserved":0,"allocated":216129536,"data_stored":215859529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,3,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525348,"kb_used_data":142420,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93846492,"statfs":{"total":96636764160,"available":96098807808,"internally_reserved":0,"allocated":145838080,"data_stored":145502580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":392097455},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":214540288,"data_stored":214536675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":144064512,"data_stored":143995406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":242814976,"data_stored":242807291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:08.258 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:08.797 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:08.797 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:08.825 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-05T13:21:07.712964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":190960142,"num_objects":46625,"num_object_clones":0,"num_object_copies":93250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46625,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244566,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10295,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160949,"ondisk_log_size":160949,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1750096,"kb_used_data":591644,"kb_used_omap":12,"kb_used_meta":1158387,"kb_avail":281365424,"statfs":{"total":289910292480,"available":288118194176,"internally_reserved":0,"allocated":605843456,"data_stored":604967947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1186188294},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,7,0,5,3,13,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18989056,"num_objects":-4636,"num_object_clones":0,"num_object_copies":-9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4636,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003143"},"pg_stats":[{"pgid":"2.f","version":"47'18004","reported_seq":18085,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.362681+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:05.362681+0000","last_peered":"2024-09-05T13:21:05.362681+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:05.362681+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12156928,"num_objects":2968,"num_object_clones":0,"num_object_copies":5936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18004,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17892","reported_seq":18001,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:03.746903+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:03.746903+0000","last_peered":"2024-09-05T13:21:03.746903+0000","last_clean":"2024-09-05T13:21:03.746903+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:03.746903+0000","last_undegraded":"2024-09-05T13:21:03.746903+0000","last_fullsized":"2024-09-05T13:21:03.746903+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15902,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18201","reported_seq":21261,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:03.758945+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:03.758945+0000","last_peered":"2024-09-05T13:21:03.758945+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:03.758945+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12263448,"num_objects":2995,"num_object_clones":0,"num_object_copies":5990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3029,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16012,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-34}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18076","reported_seq":18151,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:03.723778+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:03.723778+0000","last_peered":"2024-09-05T13:21:03.723778+0000","last_clean":"2024-09-05T13:21:03.723778+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:03.723778+0000","last_undegraded":"2024-09-05T13:21:03.723778+0000","last_fullsized":"2024-09-05T13:21:03.723778+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12083200,"num_objects":2950,"num_object_clones":0,"num_object_copies":5900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18076,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18027","reported_seq":18167,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:05.288675+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:05.288675+0000","last_peered":"2024-09-05T13:21:05.288675+0000","last_clean":"2024-09-05T13:21:05.288675+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:05.288675+0000","last_undegraded":"2024-09-05T13:21:05.288675+0000","last_fullsized":"2024-09-05T13:21:05.288675+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698176,"num_objects":2856,"num_object_clones":0,"num_object_copies":5712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14567,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18252","reported_seq":18396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.343293+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:05.343293+0000","last_peered":"2024-09-05T13:21:05.343293+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:05.343293+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11890688,"num_objects":2903,"num_object_clones":0,"num_object_copies":5806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14707,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1548}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17787","reported_seq":26819,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:04.350278+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:04.350278+0000","last_peered":"2024-09-05T13:21:04.350278+0000","last_clean":"2024-09-05T13:21:04.350278+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:04.350278+0000","last_undegraded":"2024-09-05T13:21:04.350278+0000","last_fullsized":"2024-09-05T13:21:04.350278+0000","mapping_epoch":46,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11636736,"num_objects":2841,"num_object_clones":0,"num_object_copies":5682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14106,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18181","reported_seq":18319,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:05.362855+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:05.362855+0000","last_peered":"2024-09-05T13:21:05.362855+0000","last_clean":"2024-09-05T13:21:05.362855+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:05.362855+0000","last_undegraded":"2024-09-05T13:21:05.362855+0000","last_fullsized":"2024-09-05T13:21:05.362855+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11972608,"num_objects":2923,"num_object_clones":0,"num_object_copies":5846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18041","reported_seq":18182,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:03.725703+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:03.725703+0000","last_peered":"2024-09-05T13:21:03.725703+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:03.725703+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12500992,"num_objects":3052,"num_object_clones":0,"num_object_copies":6104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15695,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17997","reported_seq":18141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.289224+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:05.289224+0000","last_peered":"2024-09-05T13:21:05.289224+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:05.289224+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11931648,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17982","reported_seq":26889,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:03.684361+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:03.684361+0000","last_peered":"2024-09-05T13:21:03.684361+0000","last_clean":"2024-09-05T13:21:03.684361+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:03.684361+0000","last_undegraded":"2024-09-05T13:21:03.684361+0000","last_fullsized":"2024-09-05T13:21:03.684361+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11997184,"num_objects":2929,"num_object_clones":0,"num_object_copies":5858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15577,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17718","reported_seq":17847,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:03.761613+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:03.761613+0000","last_peered":"2024-09-05T13:21:03.761613+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:03.761613+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11730944,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15227,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1523}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17996","reported_seq":18142,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.417285+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:05.417285+0000","last_peered":"2024-09-05T13:21:05.417285+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:05.417285+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14147,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18181","reported_seq":18374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.413499+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:05.413499+0000","last_peered":"2024-09-05T13:21:05.413499+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:05.413499+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11972608,"num_objects":2923,"num_object_clones":0,"num_object_copies":5846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18196","reported_seq":18341,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.317862+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:05.317862+0000","last_peered":"2024-09-05T13:21:05.317862+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:05.317862+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11812864,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14326,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18315","reported_seq":18461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.339974+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:05.339974+0000","last_peered":"2024-09-05T13:21:05.339974+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:05.339974+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11784675,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":190960123,"num_objects":46623,"num_object_clones":0,"num_object_copies":93246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244564,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10294,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":601419776,"data_stored":601339372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558628,"kb_used_data":238160,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93813212,"statfs":{"total":96636764160,"available":96064729088,"internally_reserved":0,"allocated":243875840,"data_stored":243605838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":328134317},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":666120,"kb_used_data":211064,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93705720,"statfs":{"total":96636764160,"available":95954657280,"internally_reserved":0,"allocated":216129536,"data_stored":215859529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,3,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525348,"kb_used_data":142420,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93846492,"statfs":{"total":96636764160,"available":96098807808,"internally_reserved":0,"allocated":145838080,"data_stored":145502580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":392097455},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":214540288,"data_stored":214536675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":144064512,"data_stored":143995406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":242814976,"data_stored":242807291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:08.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:08.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18004", 'reported_seq': 18085, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:05.362681+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:05.362681+0000', 'last_peered': '2024-09-05T13:21:05.362681+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:05.362681+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12156928, 'num_objects': 2968, 'num_object_clones': 0, 'num_object_copies': 5936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2968, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18004, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1453}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:08.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:08.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'18201", 'reported_seq': 21261, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:03.758945+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:03.758945+0000', 'last_peered': '2024-09-05T13:21:03.758945+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:03.758945+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12263448, 'num_objects': 2995, 'num_object_clones': 0, 'num_object_copies': 5990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3029, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2995, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16012, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1464, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -34}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:08.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:08.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'18252", 'reported_seq': 18396, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:05.343293+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:05.343293+0000', 'last_peered': '2024-09-05T13:21:05.343293+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:05.343293+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11890688, 'num_objects': 2903, 'num_object_clones': 0, 'num_object_copies': 5806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14707, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1548}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'18041", 'reported_seq': 18182, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:03.725703+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:03.725703+0000', 'last_peered': '2024-09-05T13:21:03.725703+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:03.725703+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12500992, 'num_objects': 3052, 'num_object_clones': 0, 'num_object_copies': 6104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15695, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'17997", 'reported_seq': 18141, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:05.289224+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:05.289224+0000', 'last_peered': '2024-09-05T13:21:05.289224+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:05.289224+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11931648, 'num_objects': 2913, 'num_object_clones': 0, 'num_object_copies': 5826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14364, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'17718", 'reported_seq': 17847, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:03.761613+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:03.761613+0000', 'last_peered': '2024-09-05T13:21:03.761613+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:03.761613+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11730944, 'num_objects': 2864, 'num_object_clones': 0, 'num_object_copies': 5728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15227, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1523}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'17996", 'reported_seq': 18142, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:05.417285+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:05.417285+0000', 'last_peered': '2024-09-05T13:21:05.417285+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:05.417285+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11804672, 'num_objects': 2882, 'num_object_clones': 0, 'num_object_copies': 5764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14147, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1537}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:08.828 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:08.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'18181", 'reported_seq': 18374, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:05.413499+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:05.413499+0000', 'last_peered': '2024-09-05T13:21:05.413499+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:05.413499+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11972608, 'num_objects': 2923, 'num_object_clones': 0, 'num_object_copies': 5846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14080, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1521}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:08.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:08.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'18196", 'reported_seq': 18341, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:05.317862+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:05.317862+0000', 'last_peered': '2024-09-05T13:21:05.317862+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:05.317862+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11812864, 'num_objects': 2884, 'num_object_clones': 0, 'num_object_copies': 5768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2884, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14326, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1682}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:08.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:08.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'18315", 'reported_seq': 18461, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:05.339974+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:05.339974+0000', 'last_peered': '2024-09-05T13:21:05.339974+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:05.339974+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11784675, 'num_objects': 2878, 'num_object_clones': 0, 'num_object_copies': 5756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14983, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1687}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:08.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:09.554 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:09.554 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43204/93250 objects degraded (46.331%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":596,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":46625,"data_bytes":190960142,"bytes_used":1792098304,"bytes_avail":288118194176,"bytes_total":289910292480,"degraded_objects":43204,"degraded_total":93250,"degraded_ratio":0.46331367292225201,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":168,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:09.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:09.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:10.115 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:10.148 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-05T13:21:09.713445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186892814,"num_objects":45632,"num_object_clones":0,"num_object_copies":91264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245559,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10393,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1734108,"kb_used_data":579808,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281381412,"statfs":{"total":289910292480,"available":288134565888,"internally_reserved":0,"allocated":593723392,"data_stored":592847883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1181928454},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,1,9,6,6,5,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23056384,"num_objects":-5629,"num_object_clones":0,"num_object_copies":-11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5629,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002970"},"pg_stats":[{"pgid":"2.f","version":"47'18004","reported_seq":18085,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.362681+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:05.362681+0000","last_peered":"2024-09-05T13:21:05.362681+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:05.362681+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12156928,"num_objects":2968,"num_object_clones":0,"num_object_copies":5936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18004,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18030","reported_seq":18140,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.646059+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:08.646059+0000","last_peered":"2024-09-05T13:21:08.646059+0000","last_clean":"2024-09-05T13:21:08.646059+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:08.646059+0000","last_undegraded":"2024-09-05T13:21:08.646059+0000","last_fullsized":"2024-09-05T13:21:08.646059+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11157504,"num_objects":2724,"num_object_clones":0,"num_object_copies":5448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16040,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18346","reported_seq":21605,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.781054+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:08.781054+0000","last_peered":"2024-09-05T13:21:08.781054+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:08.781054+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11669528,"num_objects":2850,"num_object_clones":0,"num_object_copies":5700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2850,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16157,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-81}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18207","reported_seq":18282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.760624+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:08.760624+0000","last_peered":"2024-09-05T13:21:08.760624+0000","last_clean":"2024-09-05T13:21:08.760624+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:08.760624+0000","last_undegraded":"2024-09-05T13:21:08.760624+0000","last_fullsized":"2024-09-05T13:21:08.760624+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11546624,"num_objects":2819,"num_object_clones":0,"num_object_copies":5638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18207,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18027","reported_seq":18167,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:05.288675+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:05.288675+0000","last_peered":"2024-09-05T13:21:05.288675+0000","last_clean":"2024-09-05T13:21:05.288675+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:05.288675+0000","last_undegraded":"2024-09-05T13:21:05.288675+0000","last_fullsized":"2024-09-05T13:21:05.288675+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11698176,"num_objects":2856,"num_object_clones":0,"num_object_copies":5712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14567,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18252","reported_seq":18396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.343293+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:05.343293+0000","last_peered":"2024-09-05T13:21:05.343293+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:05.343293+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11890688,"num_objects":2903,"num_object_clones":0,"num_object_copies":5806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14707,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1548}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17958","reported_seq":26990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:09.466999+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:09.466999+0000","last_peered":"2024-09-05T13:21:09.466999+0000","last_clean":"2024-09-05T13:21:09.466999+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:09.466999+0000","last_undegraded":"2024-09-05T13:21:09.466999+0000","last_fullsized":"2024-09-05T13:21:09.466999+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936320,"num_objects":2670,"num_object_clones":0,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18181","reported_seq":18319,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:05.362855+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:05.362855+0000","last_peered":"2024-09-05T13:21:05.362855+0000","last_clean":"2024-09-05T13:21:05.362855+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:05.362855+0000","last_undegraded":"2024-09-05T13:21:05.362855+0000","last_fullsized":"2024-09-05T13:21:05.362855+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11972608,"num_objects":2923,"num_object_clones":0,"num_object_copies":5846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18191","reported_seq":18332,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.739863+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:08.739863+0000","last_peered":"2024-09-05T13:21:08.739863+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:08.739863+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11886592,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15845,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1549}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17997","reported_seq":18141,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.289224+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:05.289224+0000","last_peered":"2024-09-05T13:21:05.289224+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:05.289224+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11931648,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14364,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18105","reported_seq":27012,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.258316+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:08.258316+0000","last_peered":"2024-09-05T13:21:08.258316+0000","last_clean":"2024-09-05T13:21:08.258316+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:08.258316+0000","last_undegraded":"2024-09-05T13:21:08.258316+0000","last_fullsized":"2024-09-05T13:21:08.258316+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15700,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17853","reported_seq":17982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.723693+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:08.723693+0000","last_peered":"2024-09-05T13:21:08.723693+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:08.723693+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11177984,"num_objects":2729,"num_object_clones":0,"num_object_copies":5458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15362,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17996","reported_seq":18142,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.417285+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:05.417285+0000","last_peered":"2024-09-05T13:21:05.417285+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:05.417285+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14147,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1537}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18181","reported_seq":18374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.413499+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:05.413499+0000","last_peered":"2024-09-05T13:21:05.413499+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:05.413499+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11972608,"num_objects":2923,"num_object_clones":0,"num_object_copies":5846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14080,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1521}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18196","reported_seq":18341,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.317862+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:05.317862+0000","last_peered":"2024-09-05T13:21:05.317862+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:05.317862+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11812864,"num_objects":2884,"num_object_clones":0,"num_object_copies":5768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14326,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1682}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18315","reported_seq":18461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:05.339974+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:05.339974+0000","last_peered":"2024-09-05T13:21:05.339974+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:05.339974+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11784675,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":186892795,"num_objects":45630,"num_object_clones":0,"num_object_copies":91260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45630,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245557,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10392,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589725696,"data_stored":589645292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558276,"kb_used_data":233652,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93813564,"statfs":{"total":96636764160,"available":96065089536,"internally_reserved":0,"allocated":239259648,"data_stored":238989646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":666120,"kb_used_data":211064,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93705720,"statfs":{"total":96636764160,"available":95954657280,"internally_reserved":0,"allocated":216129536,"data_stored":215859529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,3,0,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":135092,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":138334208,"data_stored":137998708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,0,1,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":214540288,"data_stored":214536675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136900608,"data_stored":136831502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238284800,"data_stored":238277115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:13.151 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:13.708 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:13.731 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-05T13:21:11.713996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":181584398,"num_objects":44336,"num_object_clones":0,"num_object_copies":88672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246855,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10393,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1714444,"kb_used_data":572624,"kb_used_omap":12,"kb_used_meta":1141747,"kb_avail":281401076,"statfs":{"total":289910292480,"available":288154701824,"internally_reserved":0,"allocated":586366976,"data_stored":585491467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1169148934},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,6,9,2,7,9,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24313856,"num_objects":-5936,"num_object_clones":0,"num_object_copies":-11872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5936,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003115"},"pg_stats":[{"pgid":"2.f","version":"47'18149","reported_seq":18230,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.352954+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:10.352954+0000","last_peered":"2024-09-05T13:21:10.352954+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:10.352954+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11563008,"num_objects":2823,"num_object_clones":0,"num_object_copies":5646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18149,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18030","reported_seq":18140,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.646059+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:08.646059+0000","last_peered":"2024-09-05T13:21:08.646059+0000","last_clean":"2024-09-05T13:21:08.646059+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:08.646059+0000","last_undegraded":"2024-09-05T13:21:08.646059+0000","last_fullsized":"2024-09-05T13:21:08.646059+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11157504,"num_objects":2724,"num_object_clones":0,"num_object_copies":5448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16040,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18346","reported_seq":21605,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.781054+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:08.781054+0000","last_peered":"2024-09-05T13:21:08.781054+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:08.781054+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11669528,"num_objects":2850,"num_object_clones":0,"num_object_copies":5700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2850,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16157,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-81}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18207","reported_seq":18282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.760624+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:08.760624+0000","last_peered":"2024-09-05T13:21:08.760624+0000","last_clean":"2024-09-05T13:21:08.760624+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:08.760624+0000","last_undegraded":"2024-09-05T13:21:08.760624+0000","last_fullsized":"2024-09-05T13:21:08.760624+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11546624,"num_objects":2819,"num_object_clones":0,"num_object_copies":5638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18207,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18170","reported_seq":18310,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:10.414391+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:10.414391+0000","last_peered":"2024-09-05T13:21:10.414391+0000","last_clean":"2024-09-05T13:21:10.414391+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:10.414391+0000","last_undegraded":"2024-09-05T13:21:10.414391+0000","last_fullsized":"2024-09-05T13:21:10.414391+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14710,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18391","reported_seq":18535,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.363446+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:10.363446+0000","last_peered":"2024-09-05T13:21:10.363446+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:10.363446+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321344,"num_objects":2764,"num_object_clones":0,"num_object_copies":5528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14846,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17958","reported_seq":26990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:09.466999+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:09.466999+0000","last_peered":"2024-09-05T13:21:09.466999+0000","last_clean":"2024-09-05T13:21:09.466999+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:09.466999+0000","last_undegraded":"2024-09-05T13:21:09.466999+0000","last_fullsized":"2024-09-05T13:21:09.466999+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936320,"num_objects":2670,"num_object_clones":0,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18325","reported_seq":18463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:10.368930+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:10.368930+0000","last_peered":"2024-09-05T13:21:10.368930+0000","last_clean":"2024-09-05T13:21:10.368930+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:10.368930+0000","last_undegraded":"2024-09-05T13:21:10.368930+0000","last_fullsized":"2024-09-05T13:21:10.368930+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11382784,"num_objects":2779,"num_object_clones":0,"num_object_copies":5558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14935,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18191","reported_seq":18332,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.739863+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:08.739863+0000","last_peered":"2024-09-05T13:21:08.739863+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:08.739863+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11886592,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15845,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1549}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18139","reported_seq":18283,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.361930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:10.361930+0000","last_peered":"2024-09-05T13:21:10.361930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:10.361930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11350016,"num_objects":2771,"num_object_clones":0,"num_object_copies":5542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14506,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1683}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18105","reported_seq":27012,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.258316+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:08.258316+0000","last_peered":"2024-09-05T13:21:08.258316+0000","last_clean":"2024-09-05T13:21:08.258316+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:08.258316+0000","last_undegraded":"2024-09-05T13:21:08.258316+0000","last_fullsized":"2024-09-05T13:21:08.258316+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15700,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17853","reported_seq":17982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.723693+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:08.723693+0000","last_peered":"2024-09-05T13:21:08.723693+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:08.723693+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11177984,"num_objects":2729,"num_object_clones":0,"num_object_copies":5458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15362,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18148","reported_seq":18294,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.339725+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:10.339725+0000","last_peered":"2024-09-05T13:21:10.339725+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:10.339725+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11182080,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14299,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1689}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18328","reported_seq":18521,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.407711+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:10.407711+0000","last_peered":"2024-09-05T13:21:10.407711+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:10.407711+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11370496,"num_objects":2776,"num_object_clones":0,"num_object_copies":5552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14227,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1668}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18352","reported_seq":18496,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.404665+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:10.404665+0000","last_peered":"2024-09-05T13:21:10.404665+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:10.404665+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11173888,"num_objects":2728,"num_object_clones":0,"num_object_copies":5456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14482,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18443","reported_seq":18589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.378647+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:10.378647+0000","last_peered":"2024-09-05T13:21:10.378647+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:10.378647+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260387,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15111,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1815}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":181584379,"num_objects":44334,"num_object_clones":0,"num_object_copies":88668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246853,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10392,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":582574080,"data_stored":582493676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558276,"kb_used_data":233652,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93813564,"statfs":{"total":96636764160,"available":96065089536,"internally_reserved":0,"allocated":239259648,"data_stored":238989646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646456,"kb_used_data":203880,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93725384,"statfs":{"total":96636764160,"available":95974793216,"internally_reserved":0,"allocated":208773120,"data_stored":208503113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":135092,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":138334208,"data_stored":137998708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,0,1,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207388672,"data_stored":207385059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136900608,"data_stored":136831502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238284800,"data_stored":238277115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:13.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:14.293 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:14.321 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-05T13:21:13.714617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":181584398,"num_objects":44336,"num_object_clones":0,"num_object_copies":88672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246855,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10393,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1714444,"kb_used_data":572624,"kb_used_omap":12,"kb_used_meta":1141747,"kb_avail":281401076,"statfs":{"total":289910292480,"available":288154701824,"internally_reserved":0,"allocated":586366976,"data_stored":585491467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1169148934},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,6,9,2,7,9,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19054592,"num_objects":-4652,"num_object_clones":0,"num_object_copies":-9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003267"},"pg_stats":[{"pgid":"2.f","version":"47'18149","reported_seq":18230,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.352954+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:10.352954+0000","last_peered":"2024-09-05T13:21:10.352954+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:10.352954+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11563008,"num_objects":2823,"num_object_clones":0,"num_object_copies":5646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18149,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18030","reported_seq":18140,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.646059+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:08.646059+0000","last_peered":"2024-09-05T13:21:08.646059+0000","last_clean":"2024-09-05T13:21:08.646059+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:08.646059+0000","last_undegraded":"2024-09-05T13:21:08.646059+0000","last_fullsized":"2024-09-05T13:21:08.646059+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11157504,"num_objects":2724,"num_object_clones":0,"num_object_copies":5448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16040,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18346","reported_seq":21605,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.781054+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:08.781054+0000","last_peered":"2024-09-05T13:21:08.781054+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:08.781054+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11669528,"num_objects":2850,"num_object_clones":0,"num_object_copies":5700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2850,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16157,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-81}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18207","reported_seq":18282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.760624+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:08.760624+0000","last_peered":"2024-09-05T13:21:08.760624+0000","last_clean":"2024-09-05T13:21:08.760624+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:08.760624+0000","last_undegraded":"2024-09-05T13:21:08.760624+0000","last_fullsized":"2024-09-05T13:21:08.760624+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11546624,"num_objects":2819,"num_object_clones":0,"num_object_copies":5638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18207,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18170","reported_seq":18310,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:10.414391+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:10.414391+0000","last_peered":"2024-09-05T13:21:10.414391+0000","last_clean":"2024-09-05T13:21:10.414391+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:10.414391+0000","last_undegraded":"2024-09-05T13:21:10.414391+0000","last_fullsized":"2024-09-05T13:21:10.414391+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14710,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18391","reported_seq":18535,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.363446+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:10.363446+0000","last_peered":"2024-09-05T13:21:10.363446+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:10.363446+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321344,"num_objects":2764,"num_object_clones":0,"num_object_copies":5528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14846,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17958","reported_seq":26990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:09.466999+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:09.466999+0000","last_peered":"2024-09-05T13:21:09.466999+0000","last_clean":"2024-09-05T13:21:09.466999+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:09.466999+0000","last_undegraded":"2024-09-05T13:21:09.466999+0000","last_fullsized":"2024-09-05T13:21:09.466999+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936320,"num_objects":2670,"num_object_clones":0,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18325","reported_seq":18463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:10.368930+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:10.368930+0000","last_peered":"2024-09-05T13:21:10.368930+0000","last_clean":"2024-09-05T13:21:10.368930+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:10.368930+0000","last_undegraded":"2024-09-05T13:21:10.368930+0000","last_fullsized":"2024-09-05T13:21:10.368930+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11382784,"num_objects":2779,"num_object_clones":0,"num_object_copies":5558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14935,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18191","reported_seq":18332,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.739863+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:08.739863+0000","last_peered":"2024-09-05T13:21:08.739863+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:08.739863+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11886592,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15845,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1549}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18139","reported_seq":18283,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.361930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:10.361930+0000","last_peered":"2024-09-05T13:21:10.361930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:10.361930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11350016,"num_objects":2771,"num_object_clones":0,"num_object_copies":5542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14506,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1683}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18105","reported_seq":27012,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.258316+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:08.258316+0000","last_peered":"2024-09-05T13:21:08.258316+0000","last_clean":"2024-09-05T13:21:08.258316+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:08.258316+0000","last_undegraded":"2024-09-05T13:21:08.258316+0000","last_fullsized":"2024-09-05T13:21:08.258316+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15700,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17853","reported_seq":17982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.723693+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:08.723693+0000","last_peered":"2024-09-05T13:21:08.723693+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:08.723693+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11177984,"num_objects":2729,"num_object_clones":0,"num_object_copies":5458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15362,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18148","reported_seq":18294,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.339725+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:10.339725+0000","last_peered":"2024-09-05T13:21:10.339725+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:10.339725+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11182080,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14299,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1689}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18328","reported_seq":18521,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.407711+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:10.407711+0000","last_peered":"2024-09-05T13:21:10.407711+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:10.407711+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11370496,"num_objects":2776,"num_object_clones":0,"num_object_copies":5552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14227,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1668}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18352","reported_seq":18496,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.404665+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:10.404665+0000","last_peered":"2024-09-05T13:21:10.404665+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:10.404665+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11173888,"num_objects":2728,"num_object_clones":0,"num_object_copies":5456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14482,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18443","reported_seq":18589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.378647+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:10.378647+0000","last_peered":"2024-09-05T13:21:10.378647+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:10.378647+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260387,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15111,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1815}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":181584379,"num_objects":44334,"num_object_clones":0,"num_object_copies":88668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246853,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10392,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":582574080,"data_stored":582493676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558276,"kb_used_data":233652,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93813564,"statfs":{"total":96636764160,"available":96065089536,"internally_reserved":0,"allocated":239259648,"data_stored":238989646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646456,"kb_used_data":203880,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93725384,"statfs":{"total":96636764160,"available":95974793216,"internally_reserved":0,"allocated":208773120,"data_stored":208503113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":135092,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":138334208,"data_stored":137998708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,0,1,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207388672,"data_stored":207385059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136900608,"data_stored":136831502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238284800,"data_stored":238277115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:14.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:14.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18149", 'reported_seq': 18230, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:10.352954+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:10.352954+0000', 'last_peered': '2024-09-05T13:21:10.352954+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:10.352954+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11563008, 'num_objects': 2823, 'num_object_clones': 0, 'num_object_copies': 5646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18149, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1598}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:14.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:14.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'18346", 'reported_seq': 21605, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:08.781054+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:08.781054+0000', 'last_peered': '2024-09-05T13:21:08.781054+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:08.781054+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11669528, 'num_objects': 2850, 'num_object_clones': 0, 'num_object_copies': 5700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2931, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2850, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16157, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1562, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -81}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:14.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:14.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'18391", 'reported_seq': 18535, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:10.363446+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:10.363446+0000', 'last_peered': '2024-09-05T13:21:10.363446+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:10.363446+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11321344, 'num_objects': 2764, 'num_object_clones': 0, 'num_object_copies': 5528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14846, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1687}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'18191", 'reported_seq': 18332, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:08.739863+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:08.739863+0000', 'last_peered': '2024-09-05T13:21:08.739863+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:08.739863+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11886592, 'num_objects': 2902, 'num_object_clones': 0, 'num_object_copies': 5804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15845, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1549}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'18139", 'reported_seq': 18283, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:10.361930+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:10.361930+0000', 'last_peered': '2024-09-05T13:21:10.361930+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:10.361930+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11350016, 'num_objects': 2771, 'num_object_clones': 0, 'num_object_copies': 5542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2771, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14506, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1683}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'17853", 'reported_seq': 17982, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:08.723693+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:08.723693+0000', 'last_peered': '2024-09-05T13:21:08.723693+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:08.723693+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11177984, 'num_objects': 2729, 'num_object_clones': 0, 'num_object_copies': 5458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15362, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1658}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'18148", 'reported_seq': 18294, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:10.339725+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:10.339725+0000', 'last_peered': '2024-09-05T13:21:10.339725+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:10.339725+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11182080, 'num_objects': 2730, 'num_object_clones': 0, 'num_object_copies': 5460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14299, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1689}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:14.323 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'18328", 'reported_seq': 18521, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:10.407711+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:10.407711+0000', 'last_peered': '2024-09-05T13:21:10.407711+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:10.407711+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11370496, 'num_objects': 2776, 'num_object_clones': 0, 'num_object_copies': 5552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14227, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1668}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:14.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:14.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'18352", 'reported_seq': 18496, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:10.404665+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:10.404665+0000', 'last_peered': '2024-09-05T13:21:10.404665+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:10.404665+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11173888, 'num_objects': 2728, 'num_object_clones': 0, 'num_object_copies': 5456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14482, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1838}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:14.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:14.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'18443", 'reported_seq': 18589, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:10.378647+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:10.378647+0000', 'last_peered': '2024-09-05T13:21:10.378647+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:10.378647+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11260387, 'num_objects': 2750, 'num_object_clones': 0, 'num_object_copies': 5500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2750, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15111, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1815}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:14.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:15.030 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:15.030 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43106/88672 objects degraded (48.613%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":601,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":44336,"data_bytes":181584398,"bytes_used":1755590656,"bytes_avail":288154701824,"bytes_total":289910292480,"degraded_objects":43106,"degraded_total":88672,"degraded_ratio":0.48612865391555393,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":180,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:15.061 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:15.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:15.644 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:15.644 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-05T13:21:13.714617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":181584398,"num_objects":44336,"num_object_clones":0,"num_object_copies":88672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246855,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10393,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1714444,"kb_used_data":572624,"kb_used_omap":12,"kb_used_meta":1141747,"kb_avail":281401076,"statfs":{"total":289910292480,"available":288154701824,"internally_reserved":0,"allocated":586366976,"data_stored":585491467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1169148934},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,6,9,2,7,9,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19054592,"num_objects":-4652,"num_object_clones":0,"num_object_copies":-9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003267"},"pg_stats":[{"pgid":"2.f","version":"47'18149","reported_seq":18230,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.352954+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:10.352954+0000","last_peered":"2024-09-05T13:21:10.352954+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:10.352954+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11563008,"num_objects":2823,"num_object_clones":0,"num_object_copies":5646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18149,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18030","reported_seq":18140,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.646059+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:08.646059+0000","last_peered":"2024-09-05T13:21:08.646059+0000","last_clean":"2024-09-05T13:21:08.646059+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:08.646059+0000","last_undegraded":"2024-09-05T13:21:08.646059+0000","last_fullsized":"2024-09-05T13:21:08.646059+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11157504,"num_objects":2724,"num_object_clones":0,"num_object_copies":5448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16040,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18346","reported_seq":21605,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.781054+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:08.781054+0000","last_peered":"2024-09-05T13:21:08.781054+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:08.781054+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11669528,"num_objects":2850,"num_object_clones":0,"num_object_copies":5700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2850,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16157,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-81}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18207","reported_seq":18282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.760624+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:08.760624+0000","last_peered":"2024-09-05T13:21:08.760624+0000","last_clean":"2024-09-05T13:21:08.760624+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:08.760624+0000","last_undegraded":"2024-09-05T13:21:08.760624+0000","last_fullsized":"2024-09-05T13:21:08.760624+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11546624,"num_objects":2819,"num_object_clones":0,"num_object_copies":5638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18207,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18170","reported_seq":18310,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:10.414391+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:10.414391+0000","last_peered":"2024-09-05T13:21:10.414391+0000","last_clean":"2024-09-05T13:21:10.414391+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:10.414391+0000","last_undegraded":"2024-09-05T13:21:10.414391+0000","last_fullsized":"2024-09-05T13:21:10.414391+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14710,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18391","reported_seq":18535,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.363446+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:10.363446+0000","last_peered":"2024-09-05T13:21:10.363446+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:10.363446+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321344,"num_objects":2764,"num_object_clones":0,"num_object_copies":5528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14846,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17958","reported_seq":26990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:09.466999+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:09.466999+0000","last_peered":"2024-09-05T13:21:09.466999+0000","last_clean":"2024-09-05T13:21:09.466999+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:09.466999+0000","last_undegraded":"2024-09-05T13:21:09.466999+0000","last_fullsized":"2024-09-05T13:21:09.466999+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936320,"num_objects":2670,"num_object_clones":0,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14277,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18325","reported_seq":18463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:10.368930+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:10.368930+0000","last_peered":"2024-09-05T13:21:10.368930+0000","last_clean":"2024-09-05T13:21:10.368930+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:10.368930+0000","last_undegraded":"2024-09-05T13:21:10.368930+0000","last_fullsized":"2024-09-05T13:21:10.368930+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11382784,"num_objects":2779,"num_object_clones":0,"num_object_copies":5558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14935,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18191","reported_seq":18332,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.739863+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:08.739863+0000","last_peered":"2024-09-05T13:21:08.739863+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:08.739863+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11886592,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15845,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1549}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18139","reported_seq":18283,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.361930+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:10.361930+0000","last_peered":"2024-09-05T13:21:10.361930+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:10.361930+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11350016,"num_objects":2771,"num_object_clones":0,"num_object_copies":5542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14506,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1683}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18105","reported_seq":27012,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:08.258316+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:08.258316+0000","last_peered":"2024-09-05T13:21:08.258316+0000","last_clean":"2024-09-05T13:21:08.258316+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:08.258316+0000","last_undegraded":"2024-09-05T13:21:08.258316+0000","last_fullsized":"2024-09-05T13:21:08.258316+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15700,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17853","reported_seq":17982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:08.723693+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:08.723693+0000","last_peered":"2024-09-05T13:21:08.723693+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:08.723693+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11177984,"num_objects":2729,"num_object_clones":0,"num_object_copies":5458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15362,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18148","reported_seq":18294,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.339725+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:10.339725+0000","last_peered":"2024-09-05T13:21:10.339725+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:10.339725+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11182080,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14299,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1689}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18328","reported_seq":18521,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.407711+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:10.407711+0000","last_peered":"2024-09-05T13:21:10.407711+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:10.407711+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11370496,"num_objects":2776,"num_object_clones":0,"num_object_copies":5552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14227,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1668}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18352","reported_seq":18496,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.404665+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:10.404665+0000","last_peered":"2024-09-05T13:21:10.404665+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:10.404665+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11173888,"num_objects":2728,"num_object_clones":0,"num_object_copies":5456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14482,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18443","reported_seq":18589,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:10.378647+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:10.378647+0000","last_peered":"2024-09-05T13:21:10.378647+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:10.378647+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11260387,"num_objects":2750,"num_object_clones":0,"num_object_copies":5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15111,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1815}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":181584379,"num_objects":44334,"num_object_clones":0,"num_object_copies":88668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246853,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10392,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":582574080,"data_stored":582493676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558276,"kb_used_data":233652,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93813564,"statfs":{"total":96636764160,"available":96065089536,"internally_reserved":0,"allocated":239259648,"data_stored":238989646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":332394157},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646456,"kb_used_data":203880,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93725384,"statfs":{"total":96636764160,"available":95974793216,"internally_reserved":0,"allocated":208773120,"data_stored":208503113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":135092,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":138334208,"data_stored":137998708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,0,1,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207388672,"data_stored":207385059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":136900608,"data_stored":136831502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":238284800,"data_stored":238277115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:18.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:19.214 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:19.216 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:19.235 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-05T13:21:17.716016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":172245518,"num_objects":42056,"num_object_clones":0,"num_object_copies":84112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249135,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10495,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1711568,"kb_used_data":553120,"kb_used_omap":12,"kb_used_meta":1158387,"kb_avail":281403952,"statfs":{"total":289910292480,"available":288157646848,"internally_reserved":0,"allocated":566394880,"data_stored":565519371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1186188294},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,9,4,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18714624,"num_objects":-4569,"num_object_clones":0,"num_object_copies":-9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4569,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003721"},"pg_stats":[{"pgid":"2.f","version":"47'18303","reported_seq":18384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.393879+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:15.393879+0000","last_peered":"2024-09-05T13:21:15.393879+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:15.393879+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18303,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1752}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18162","reported_seq":18272,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:13.755771+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:13.755771+0000","last_peered":"2024-09-05T13:21:13.755771+0000","last_clean":"2024-09-05T13:21:13.755771+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:13.755771+0000","last_undegraded":"2024-09-05T13:21:13.755771+0000","last_fullsized":"2024-09-05T13:21:13.755771+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10616832,"num_objects":2592,"num_object_clones":0,"num_object_copies":5184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16172,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18489","reported_seq":21952,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:13.758561+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:13.758561+0000","last_peered":"2024-09-05T13:21:13.758561+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:13.758561+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11083800,"num_objects":2707,"num_object_clones":0,"num_object_copies":5414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2707,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16300,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1664,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18342","reported_seq":18417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:13.620080+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:13.620080+0000","last_peered":"2024-09-05T13:21:13.620080+0000","last_clean":"2024-09-05T13:21:13.620080+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:13.620080+0000","last_undegraded":"2024-09-05T13:21:13.620080+0000","last_fullsized":"2024-09-05T13:21:13.620080+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10993664,"num_objects":2684,"num_object_clones":0,"num_object_copies":5368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18342,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18320","reported_seq":18460,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:15.277885+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:15.277885+0000","last_peered":"2024-09-05T13:21:15.277885+0000","last_clean":"2024-09-05T13:21:15.277885+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:15.277885+0000","last_undegraded":"2024-09-05T13:21:15.277885+0000","last_fullsized":"2024-09-05T13:21:15.277885+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10498048,"num_objects":2563,"num_object_clones":0,"num_object_copies":5126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14860,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18525","reported_seq":18669,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.438159+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:15.438159+0000","last_peered":"2024-09-05T13:21:15.438159+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:15.438159+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10772480,"num_objects":2630,"num_object_clones":0,"num_object_copies":5260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18086","reported_seq":27118,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:14.584590+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:14.584590+0000","last_peered":"2024-09-05T13:21:14.584590+0000","last_clean":"2024-09-05T13:21:14.584590+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:14.584590+0000","last_undegraded":"2024-09-05T13:21:14.584590+0000","last_fullsized":"2024-09-05T13:21:14.584590+0000","mapping_epoch":46,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10412032,"num_objects":2542,"num_object_clones":0,"num_object_copies":5084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14405,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18484","reported_seq":18622,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:15.330941+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:15.330941+0000","last_peered":"2024-09-05T13:21:15.330941+0000","last_clean":"2024-09-05T13:21:15.330941+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:15.330941+0000","last_undegraded":"2024-09-05T13:21:15.330941+0000","last_fullsized":"2024-09-05T13:21:15.330941+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10731520,"num_objects":2620,"num_object_clones":0,"num_object_copies":5240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15094,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106234+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106234+0000","last_peered":"2024-09-05T13:19:36.106234+0000","last_clean":"2024-09-05T13:19:36.106234+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106234+0000","last_undegraded":"2024-09-05T13:19:36.106234+0000","last_fullsized":"2024-09-05T13:19:36.106234+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18336","reported_seq":18477,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:13.791208+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:13.791208+0000","last_peered":"2024-09-05T13:21:13.791208+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:13.791208+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11292672,"num_objects":2757,"num_object_clones":0,"num_object_copies":5514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15990,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1694}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18282","reported_seq":18426,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.436309+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:15.436309+0000","last_peered":"2024-09-05T13:21:15.436309+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:15.436309+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10764288,"num_objects":2628,"num_object_clones":0,"num_object_copies":5256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14649,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1826}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18242","reported_seq":27149,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:13.688185+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:13.688185+0000","last_peered":"2024-09-05T13:21:13.688185+0000","last_clean":"2024-09-05T13:21:13.688185+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:13.688185+0000","last_undegraded":"2024-09-05T13:21:13.688185+0000","last_fullsized":"2024-09-05T13:21:13.688185+0000","mapping_epoch":44,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15837,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17991","reported_seq":18120,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:13.724030+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:13.724030+0000","last_peered":"2024-09-05T13:21:13.724030+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:13.724030+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10612736,"num_objects":2591,"num_object_clones":0,"num_object_copies":5182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15500,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1796}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18310","reported_seq":18454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.388325+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:15.388325+0000","last_peered":"2024-09-05T13:21:15.388325+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:15.388325+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14461,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18458","reported_seq":18650,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.378526+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:15.378526+0000","last_peered":"2024-09-05T13:21:15.378526+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:15.378526+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10838016,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14357,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18508","reported_seq":18653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.258778+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:15.258778+0000","last_peered":"2024-09-05T13:21:15.258778+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:15.258778+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10534912,"num_objects":2572,"num_object_clones":0,"num_object_copies":5144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14638,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18577","reported_seq":18723,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.386126+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:15.386126+0000","last_peered":"2024-09-05T13:21:15.386126+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:15.386126+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10711523,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15245,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":172245499,"num_objects":42054,"num_object_clones":0,"num_object_copies":84108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42054,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249133,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10494,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":563585024,"data_stored":563504620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557912,"kb_used_data":229128,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93813928,"statfs":{"total":96636764160,"available":96065462272,"internally_reserved":0,"allocated":234627072,"data_stored":234357070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":336653997},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,0,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647836,"kb_used_data":196940,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93724004,"statfs":{"total":96636764160,"available":95973380096,"internally_reserved":0,"allocated":201666560,"data_stored":201396553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505820,"kb_used_data":127052,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93866020,"statfs":{"total":96636764160,"available":96118804480,"internally_reserved":0,"allocated":130101248,"data_stored":129765748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200331264,"data_stored":200327651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":129703936,"data_stored":129634830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":233549824,"data_stored":233542139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:19.237 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:19.778 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:19.778 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:19.800 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-05T13:21:19.716776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":168088078,"num_objects":41041,"num_object_clones":0,"num_object_copies":82082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250150,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10576,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1697368,"kb_used_data":541668,"kb_used_omap":12,"kb_used_meta":1155635,"kb_avail":281418152,"statfs":{"total":289910292480,"available":288172187648,"internally_reserved":0,"allocated":554668032,"data_stored":553792523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1183370246},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,4,13,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22872064,"num_objects":-5584,"num_object_clones":0,"num_object_copies":-11168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5584,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003811"},"pg_stats":[{"pgid":"2.f","version":"47'18303","reported_seq":18384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.393879+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:15.393879+0000","last_peered":"2024-09-05T13:21:15.393879+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:15.393879+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18303,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1752}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18308","reported_seq":18418,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.728310+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:18.728310+0000","last_peered":"2024-09-05T13:21:18.728310+0000","last_clean":"2024-09-05T13:21:18.728310+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:18.728310+0000","last_undegraded":"2024-09-05T13:21:18.728310+0000","last_fullsized":"2024-09-05T13:21:18.728310+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10018816,"num_objects":2446,"num_object_clones":0,"num_object_copies":4892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16318,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18652","reported_seq":22276,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.727279+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:18.727279+0000","last_peered":"2024-09-05T13:21:18.727279+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:18.727279+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10416152,"num_objects":2544,"num_object_clones":0,"num_object_copies":5088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2748,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16463,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18478","reported_seq":18553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.798982+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:18.798982+0000","last_peered":"2024-09-05T13:21:18.798982+0000","last_clean":"2024-09-05T13:21:18.798982+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:18.798982+0000","last_undegraded":"2024-09-05T13:21:18.798982+0000","last_fullsized":"2024-09-05T13:21:18.798982+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10436608,"num_objects":2548,"num_object_clones":0,"num_object_copies":5096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18478,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18320","reported_seq":18460,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:15.277885+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:15.277885+0000","last_peered":"2024-09-05T13:21:15.277885+0000","last_clean":"2024-09-05T13:21:15.277885+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:15.277885+0000","last_undegraded":"2024-09-05T13:21:15.277885+0000","last_fullsized":"2024-09-05T13:21:15.277885+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10498048,"num_objects":2563,"num_object_clones":0,"num_object_copies":5126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14860,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18525","reported_seq":18669,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.438159+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:15.438159+0000","last_peered":"2024-09-05T13:21:15.438159+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:15.438159+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10772480,"num_objects":2630,"num_object_clones":0,"num_object_copies":5260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18221","reported_seq":27253,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:19.624328+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:19.624328+0000","last_peered":"2024-09-05T13:21:19.624328+0000","last_clean":"2024-09-05T13:21:19.624328+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:19.624328+0000","last_undegraded":"2024-09-05T13:21:19.624328+0000","last_fullsized":"2024-09-05T13:21:19.624328+0000","mapping_epoch":46,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9859072,"num_objects":2407,"num_object_clones":0,"num_object_copies":4814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14540,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18484","reported_seq":18622,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:15.330941+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:15.330941+0000","last_peered":"2024-09-05T13:21:15.330941+0000","last_clean":"2024-09-05T13:21:15.330941+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:15.330941+0000","last_undegraded":"2024-09-05T13:21:15.330941+0000","last_fullsized":"2024-09-05T13:21:15.330941+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10731520,"num_objects":2620,"num_object_clones":0,"num_object_copies":5240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15094,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18472","reported_seq":18613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.742613+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:18.742613+0000","last_peered":"2024-09-05T13:21:18.742613+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:18.742613+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10735616,"num_objects":2621,"num_object_clones":0,"num_object_copies":5242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16126,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18282","reported_seq":18426,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.436309+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:15.436309+0000","last_peered":"2024-09-05T13:21:15.436309+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:15.436309+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10764288,"num_objects":2628,"num_object_clones":0,"num_object_copies":5256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14649,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1826}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18392","reported_seq":27299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.668468+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:18.668468+0000","last_peered":"2024-09-05T13:21:18.668468+0000","last_clean":"2024-09-05T13:21:18.668468+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:18.668468+0000","last_undegraded":"2024-09-05T13:21:18.668468+0000","last_fullsized":"2024-09-05T13:21:18.668468+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15987,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18140","reported_seq":18269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.749736+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:18.749736+0000","last_peered":"2024-09-05T13:21:18.749736+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:18.749736+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002432,"num_objects":2442,"num_object_clones":0,"num_object_copies":4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15649,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1945}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18310","reported_seq":18454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.388325+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:15.388325+0000","last_peered":"2024-09-05T13:21:15.388325+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:15.388325+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14461,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18458","reported_seq":18650,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.378526+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:15.378526+0000","last_peered":"2024-09-05T13:21:15.378526+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:15.378526+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10838016,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14357,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18508","reported_seq":18653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.258778+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:15.258778+0000","last_peered":"2024-09-05T13:21:15.258778+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:15.258778+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10534912,"num_objects":2572,"num_object_clones":0,"num_object_copies":5144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14638,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18577","reported_seq":18723,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.386126+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:15.386126+0000","last_peered":"2024-09-05T13:21:15.386126+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:15.386126+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10711523,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15245,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":168088059,"num_objects":41039,"num_object_clones":0,"num_object_copies":82078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250148,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10575,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":551571456,"data_stored":551491052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540760,"kb_used_data":224456,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93831080,"statfs":{"total":96636764160,"available":96083025920,"internally_reserved":0,"allocated":229842944,"data_stored":229572942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323874477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647836,"kb_used_data":196940,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93724004,"statfs":{"total":96636764160,"available":95973380096,"internally_reserved":0,"allocated":201666560,"data_stored":201396553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508772,"kb_used_data":120272,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93863068,"statfs":{"total":96636764160,"available":96115781632,"internally_reserved":0,"allocated":123158528,"data_stored":122823028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":397799087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200331264,"data_stored":200327651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122490880,"data_stored":122421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228749312,"data_stored":228741627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:19.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:19.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18303", 'reported_seq': 18384, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:15.393879+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:15.393879+0000', 'last_peered': '2024-09-05T13:21:15.393879+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:15.393879+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10932224, 'num_objects': 2669, 'num_object_clones': 0, 'num_object_copies': 5338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2669, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18303, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1752}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:19.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:19.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'18652", 'reported_seq': 22276, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:18.727279+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:18.727279+0000', 'last_peered': '2024-09-05T13:21:18.727279+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:18.727279+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10416152, 'num_objects': 2544, 'num_object_clones': 0, 'num_object_copies': 5088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2748, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2544, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16463, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1745, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -204}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:19.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:19.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'18525", 'reported_seq': 18669, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:15.438159+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:15.438159+0000', 'last_peered': '2024-09-05T13:21:15.438159+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:15.438159+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10772480, 'num_objects': 2630, 'num_object_clones': 0, 'num_object_copies': 5260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14980, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1821}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:19.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:19.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'18472", 'reported_seq': 18613, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:18.742613+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:18.742613+0000', 'last_peered': '2024-09-05T13:21:18.742613+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:18.742613+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10735616, 'num_objects': 2621, 'num_object_clones': 0, 'num_object_copies': 5242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16126, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1830}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:19.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:19.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'18282", 'reported_seq': 18426, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:15.436309+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:15.436309+0000', 'last_peered': '2024-09-05T13:21:15.436309+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:15.436309+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10764288, 'num_objects': 2628, 'num_object_clones': 0, 'num_object_copies': 5256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2628, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14649, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1826}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'18140", 'reported_seq': 18269, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:18.749736+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:18.749736+0000', 'last_peered': '2024-09-05T13:21:18.749736+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:18.749736+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10002432, 'num_objects': 2442, 'num_object_clones': 0, 'num_object_copies': 4884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15649, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1945}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'18310", 'reported_seq': 18454, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:15.388325+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:15.388325+0000', 'last_peered': '2024-09-05T13:21:15.388325+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:15.388325+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10518528, 'num_objects': 2568, 'num_object_clones': 0, 'num_object_copies': 5136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14461, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1851}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'18458", 'reported_seq': 18650, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:15.378526+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:15.378526+0000', 'last_peered': '2024-09-05T13:21:15.378526+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:15.378526+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10838016, 'num_objects': 2646, 'num_object_clones': 0, 'num_object_copies': 5292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14357, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'18508", 'reported_seq': 18653, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:15.258778+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:15.258778+0000', 'last_peered': '2024-09-05T13:21:15.258778+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:15.258778+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10534912, 'num_objects': 2572, 'num_object_clones': 0, 'num_object_copies': 5144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14638, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1994}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:19.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'18577", 'reported_seq': 18723, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:15.386126+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:15.386126+0000', 'last_peered': '2024-09-05T13:21:15.386126+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:15.386126+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10711523, 'num_objects': 2616, 'num_object_clones': 0, 'num_object_copies': 5232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15245, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:19.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:20.475 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:20.475 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42923/82082 objects degraded (52.293%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":607,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":41041,"data_bytes":168088078,"bytes_used":1738104832,"bytes_avail":288172187648,"bytes_total":289910292480,"degraded_objects":42923,"degraded_total":82082,"degraded_ratio":0.52292829122097417,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":465},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:20.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:20.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:21.078 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:21.079 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:21.103 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-05T13:21:19.716776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":168088078,"num_objects":41041,"num_object_clones":0,"num_object_copies":82082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250150,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10576,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1697368,"kb_used_data":541668,"kb_used_omap":12,"kb_used_meta":1155635,"kb_avail":281418152,"statfs":{"total":289910292480,"available":288172187648,"internally_reserved":0,"allocated":554668032,"data_stored":553792523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1183370246},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,4,13,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22872064,"num_objects":-5584,"num_object_clones":0,"num_object_copies":-11168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5584,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003811"},"pg_stats":[{"pgid":"2.f","version":"47'18303","reported_seq":18384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.393879+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:15.393879+0000","last_peered":"2024-09-05T13:21:15.393879+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:15.393879+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932224,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18303,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1752}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18308","reported_seq":18418,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.728310+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:18.728310+0000","last_peered":"2024-09-05T13:21:18.728310+0000","last_clean":"2024-09-05T13:21:18.728310+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:18.728310+0000","last_undegraded":"2024-09-05T13:21:18.728310+0000","last_fullsized":"2024-09-05T13:21:18.728310+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10018816,"num_objects":2446,"num_object_clones":0,"num_object_copies":4892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16318,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18652","reported_seq":22276,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.727279+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:18.727279+0000","last_peered":"2024-09-05T13:21:18.727279+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:18.727279+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10416152,"num_objects":2544,"num_object_clones":0,"num_object_copies":5088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2748,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16463,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18478","reported_seq":18553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.798982+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:18.798982+0000","last_peered":"2024-09-05T13:21:18.798982+0000","last_clean":"2024-09-05T13:21:18.798982+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:18.798982+0000","last_undegraded":"2024-09-05T13:21:18.798982+0000","last_fullsized":"2024-09-05T13:21:18.798982+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10436608,"num_objects":2548,"num_object_clones":0,"num_object_copies":5096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18478,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18320","reported_seq":18460,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:15.277885+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:15.277885+0000","last_peered":"2024-09-05T13:21:15.277885+0000","last_clean":"2024-09-05T13:21:15.277885+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:15.277885+0000","last_undegraded":"2024-09-05T13:21:15.277885+0000","last_fullsized":"2024-09-05T13:21:15.277885+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10498048,"num_objects":2563,"num_object_clones":0,"num_object_copies":5126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14860,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18525","reported_seq":18669,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.438159+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:15.438159+0000","last_peered":"2024-09-05T13:21:15.438159+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:15.438159+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10772480,"num_objects":2630,"num_object_clones":0,"num_object_copies":5260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18221","reported_seq":27253,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:19.624328+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:19.624328+0000","last_peered":"2024-09-05T13:21:19.624328+0000","last_clean":"2024-09-05T13:21:19.624328+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:19.624328+0000","last_undegraded":"2024-09-05T13:21:19.624328+0000","last_fullsized":"2024-09-05T13:21:19.624328+0000","mapping_epoch":46,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9859072,"num_objects":2407,"num_object_clones":0,"num_object_copies":4814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14540,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18484","reported_seq":18622,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:15.330941+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:15.330941+0000","last_peered":"2024-09-05T13:21:15.330941+0000","last_clean":"2024-09-05T13:21:15.330941+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:15.330941+0000","last_undegraded":"2024-09-05T13:21:15.330941+0000","last_fullsized":"2024-09-05T13:21:15.330941+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10731520,"num_objects":2620,"num_object_clones":0,"num_object_copies":5240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15094,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18472","reported_seq":18613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.742613+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:18.742613+0000","last_peered":"2024-09-05T13:21:18.742613+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:18.742613+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10735616,"num_objects":2621,"num_object_clones":0,"num_object_copies":5242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16126,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18282","reported_seq":18426,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.436309+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:15.436309+0000","last_peered":"2024-09-05T13:21:15.436309+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:15.436309+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10764288,"num_objects":2628,"num_object_clones":0,"num_object_copies":5256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14649,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1826}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18392","reported_seq":27299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.668468+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:18.668468+0000","last_peered":"2024-09-05T13:21:18.668468+0000","last_clean":"2024-09-05T13:21:18.668468+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:18.668468+0000","last_undegraded":"2024-09-05T13:21:18.668468+0000","last_fullsized":"2024-09-05T13:21:18.668468+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15987,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18140","reported_seq":18269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.749736+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:18.749736+0000","last_peered":"2024-09-05T13:21:18.749736+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:18.749736+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002432,"num_objects":2442,"num_object_clones":0,"num_object_copies":4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15649,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1945}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18310","reported_seq":18454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.388325+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:15.388325+0000","last_peered":"2024-09-05T13:21:15.388325+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:15.388325+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10518528,"num_objects":2568,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14461,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18458","reported_seq":18650,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.378526+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:15.378526+0000","last_peered":"2024-09-05T13:21:15.378526+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:15.378526+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10838016,"num_objects":2646,"num_object_clones":0,"num_object_copies":5292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14357,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18508","reported_seq":18653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.258778+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:15.258778+0000","last_peered":"2024-09-05T13:21:15.258778+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:15.258778+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10534912,"num_objects":2572,"num_object_clones":0,"num_object_copies":5144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14638,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1994}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18577","reported_seq":18723,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:15.386126+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:15.386126+0000","last_peered":"2024-09-05T13:21:15.386126+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:15.386126+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10711523,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15245,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":168088059,"num_objects":41039,"num_object_clones":0,"num_object_copies":82078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41039,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250148,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10575,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":551571456,"data_stored":551491052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540760,"kb_used_data":224456,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93831080,"statfs":{"total":96636764160,"available":96083025920,"internally_reserved":0,"allocated":229842944,"data_stored":229572942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323874477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647836,"kb_used_data":196940,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93724004,"statfs":{"total":96636764160,"available":95973380096,"internally_reserved":0,"allocated":201666560,"data_stored":201396553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508772,"kb_used_data":120272,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93863068,"statfs":{"total":96636764160,"available":96115781632,"internally_reserved":0,"allocated":123158528,"data_stored":122823028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":397799087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200331264,"data_stored":200327651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122490880,"data_stored":122421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228749312,"data_stored":228741627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:24.105 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:24.695 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:24.697 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:24.720 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-05T13:21:23.718389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":162628110,"num_objects":39708,"num_object_clones":0,"num_object_copies":79416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39708,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251483,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10576,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1694656,"kb_used_data":534796,"kb_used_omap":12,"kb_used_meta":1159795,"kb_avail":281420864,"statfs":{"total":289910292480,"available":288174964736,"internally_reserved":0,"allocated":547631104,"data_stored":546755595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1187630086},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,2,16,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18956288,"num_objects":-4628,"num_object_clones":0,"num_object_copies":-9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4628,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004392"},"pg_stats":[{"pgid":"2.f","version":"47'18439","reported_seq":18520,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.445043+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:20.445043+0000","last_peered":"2024-09-05T13:21:20.445043+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:20.445043+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10375168,"num_objects":2533,"num_object_clones":0,"num_object_copies":5066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18439,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1888}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18308","reported_seq":18418,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.728310+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:18.728310+0000","last_peered":"2024-09-05T13:21:18.728310+0000","last_clean":"2024-09-05T13:21:18.728310+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:18.728310+0000","last_undegraded":"2024-09-05T13:21:18.728310+0000","last_fullsized":"2024-09-05T13:21:18.728310+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10018816,"num_objects":2446,"num_object_clones":0,"num_object_copies":4892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16318,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18652","reported_seq":22276,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.727279+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:18.727279+0000","last_peered":"2024-09-05T13:21:18.727279+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:18.727279+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10416152,"num_objects":2544,"num_object_clones":0,"num_object_copies":5088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2748,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16463,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18478","reported_seq":18553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.798982+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:18.798982+0000","last_peered":"2024-09-05T13:21:18.798982+0000","last_clean":"2024-09-05T13:21:18.798982+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:18.798982+0000","last_undegraded":"2024-09-05T13:21:18.798982+0000","last_fullsized":"2024-09-05T13:21:18.798982+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10436608,"num_objects":2548,"num_object_clones":0,"num_object_copies":5096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18478,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18452","reported_seq":18592,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:20.299800+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:20.299800+0000","last_peered":"2024-09-05T13:21:20.299800+0000","last_clean":"2024-09-05T13:21:20.299800+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:20.299800+0000","last_undegraded":"2024-09-05T13:21:20.299800+0000","last_fullsized":"2024-09-05T13:21:20.299800+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14992,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18672","reported_seq":18816,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.520593+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:20.520593+0000","last_peered":"2024-09-05T13:21:20.520593+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:20.520593+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10170368,"num_objects":2483,"num_object_clones":0,"num_object_copies":4966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15127,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18221","reported_seq":27253,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:19.624328+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:19.624328+0000","last_peered":"2024-09-05T13:21:19.624328+0000","last_clean":"2024-09-05T13:21:19.624328+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:19.624328+0000","last_undegraded":"2024-09-05T13:21:19.624328+0000","last_fullsized":"2024-09-05T13:21:19.624328+0000","mapping_epoch":46,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9859072,"num_objects":2407,"num_object_clones":0,"num_object_copies":4814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14540,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18638","reported_seq":18776,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:20.458943+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:20.458943+0000","last_peered":"2024-09-05T13:21:20.458943+0000","last_clean":"2024-09-05T13:21:20.458943+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:20.458943+0000","last_undegraded":"2024-09-05T13:21:20.458943+0000","last_fullsized":"2024-09-05T13:21:20.458943+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10100736,"num_objects":2466,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15248,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18472","reported_seq":18613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.742613+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:18.742613+0000","last_peered":"2024-09-05T13:21:18.742613+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:18.742613+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10735616,"num_objects":2621,"num_object_clones":0,"num_object_copies":5242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16126,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18446","reported_seq":18590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.491429+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:20.491429+0000","last_peered":"2024-09-05T13:21:20.491429+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:20.491429+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10092544,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14813,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1990}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18392","reported_seq":27299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.668468+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:18.668468+0000","last_peered":"2024-09-05T13:21:18.668468+0000","last_clean":"2024-09-05T13:21:18.668468+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:18.668468+0000","last_undegraded":"2024-09-05T13:21:18.668468+0000","last_fullsized":"2024-09-05T13:21:18.668468+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15987,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18140","reported_seq":18269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.749736+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:18.749736+0000","last_peered":"2024-09-05T13:21:18.749736+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:18.749736+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002432,"num_objects":2442,"num_object_clones":0,"num_object_copies":4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15649,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1945}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18429","reported_seq":18575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.456035+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:20.456035+0000","last_peered":"2024-09-05T13:21:20.456035+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:20.456035+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10031104,"num_objects":2449,"num_object_clones":0,"num_object_copies":4898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1970}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18614","reported_seq":18807,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.500877+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:20.500877+0000","last_peered":"2024-09-05T13:21:20.500877+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:20.500877+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10199040,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14513,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1954}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18679","reported_seq":18820,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.494534+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:20.494534+0000","last_peered":"2024-09-05T13:21:20.494534+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:20.494534+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9834496,"num_objects":2401,"num_object_clones":0,"num_object_copies":4802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14809,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18731","reported_seq":18877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.504497+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:20.504497+0000","last_peered":"2024-09-05T13:21:20.504497+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:20.504497+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10080739,"num_objects":2462,"num_object_clones":0,"num_object_copies":4924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15399,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":162628091,"num_objects":39706,"num_object_clones":0,"num_object_copies":79412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39706,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251481,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10575,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544509952,"data_stored":544429548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540760,"kb_used_data":224456,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93831080,"statfs":{"total":96636764160,"available":96083025920,"internally_reserved":0,"allocated":229842944,"data_stored":229572942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323874477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645124,"kb_used_data":190068,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93726716,"statfs":{"total":96636764160,"available":95976157184,"internally_reserved":0,"allocated":194629632,"data_stored":194359625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,0,9,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508772,"kb_used_data":120272,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93863068,"statfs":{"total":96636764160,"available":96115781632,"internally_reserved":0,"allocated":123158528,"data_stored":122823028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":397799087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":193269760,"data_stored":193266147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122490880,"data_stored":122421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228749312,"data_stored":228741627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:24.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:25.281 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:25.312 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-05T13:21:23.718389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":162628110,"num_objects":39708,"num_object_clones":0,"num_object_copies":79416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39708,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251483,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10576,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1694656,"kb_used_data":534796,"kb_used_omap":12,"kb_used_meta":1159795,"kb_avail":281420864,"statfs":{"total":289910292480,"available":288174964736,"internally_reserved":0,"allocated":547631104,"data_stored":546755595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1187630086},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,2,16,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18956288,"num_objects":-4628,"num_object_clones":0,"num_object_copies":-9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4628,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004392"},"pg_stats":[{"pgid":"2.f","version":"47'18439","reported_seq":18520,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.445043+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:20.445043+0000","last_peered":"2024-09-05T13:21:20.445043+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:20.445043+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10375168,"num_objects":2533,"num_object_clones":0,"num_object_copies":5066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18439,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1888}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18308","reported_seq":18418,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.728310+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:18.728310+0000","last_peered":"2024-09-05T13:21:18.728310+0000","last_clean":"2024-09-05T13:21:18.728310+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:18.728310+0000","last_undegraded":"2024-09-05T13:21:18.728310+0000","last_fullsized":"2024-09-05T13:21:18.728310+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10018816,"num_objects":2446,"num_object_clones":0,"num_object_copies":4892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16318,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18652","reported_seq":22276,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.727279+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:18.727279+0000","last_peered":"2024-09-05T13:21:18.727279+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:18.727279+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10416152,"num_objects":2544,"num_object_clones":0,"num_object_copies":5088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2748,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16463,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18478","reported_seq":18553,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.798982+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:18.798982+0000","last_peered":"2024-09-05T13:21:18.798982+0000","last_clean":"2024-09-05T13:21:18.798982+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:18.798982+0000","last_undegraded":"2024-09-05T13:21:18.798982+0000","last_fullsized":"2024-09-05T13:21:18.798982+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10436608,"num_objects":2548,"num_object_clones":0,"num_object_copies":5096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18478,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18452","reported_seq":18592,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:20.299800+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:20.299800+0000","last_peered":"2024-09-05T13:21:20.299800+0000","last_clean":"2024-09-05T13:21:20.299800+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:20.299800+0000","last_undegraded":"2024-09-05T13:21:20.299800+0000","last_fullsized":"2024-09-05T13:21:20.299800+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957376,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14992,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18672","reported_seq":18816,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.520593+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:20.520593+0000","last_peered":"2024-09-05T13:21:20.520593+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:20.520593+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10170368,"num_objects":2483,"num_object_clones":0,"num_object_copies":4966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15127,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18221","reported_seq":27253,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:19.624328+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:19.624328+0000","last_peered":"2024-09-05T13:21:19.624328+0000","last_clean":"2024-09-05T13:21:19.624328+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:19.624328+0000","last_undegraded":"2024-09-05T13:21:19.624328+0000","last_fullsized":"2024-09-05T13:21:19.624328+0000","mapping_epoch":46,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9859072,"num_objects":2407,"num_object_clones":0,"num_object_copies":4814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14540,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18638","reported_seq":18776,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:20.458943+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:20.458943+0000","last_peered":"2024-09-05T13:21:20.458943+0000","last_clean":"2024-09-05T13:21:20.458943+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:20.458943+0000","last_undegraded":"2024-09-05T13:21:20.458943+0000","last_fullsized":"2024-09-05T13:21:20.458943+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10100736,"num_objects":2466,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15248,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18472","reported_seq":18613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.742613+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:18.742613+0000","last_peered":"2024-09-05T13:21:18.742613+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:18.742613+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10735616,"num_objects":2621,"num_object_clones":0,"num_object_copies":5242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16126,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18446","reported_seq":18590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.491429+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:20.491429+0000","last_peered":"2024-09-05T13:21:20.491429+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:20.491429+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10092544,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14813,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1990}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18392","reported_seq":27299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:18.668468+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:18.668468+0000","last_peered":"2024-09-05T13:21:18.668468+0000","last_clean":"2024-09-05T13:21:18.668468+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:18.668468+0000","last_undegraded":"2024-09-05T13:21:18.668468+0000","last_fullsized":"2024-09-05T13:21:18.668468+0000","mapping_epoch":44,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15987,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18140","reported_seq":18269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:18.749736+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:18.749736+0000","last_peered":"2024-09-05T13:21:18.749736+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:18.749736+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002432,"num_objects":2442,"num_object_clones":0,"num_object_copies":4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15649,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1945}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18429","reported_seq":18575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.456035+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:20.456035+0000","last_peered":"2024-09-05T13:21:20.456035+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:20.456035+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10031104,"num_objects":2449,"num_object_clones":0,"num_object_copies":4898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1970}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18614","reported_seq":18807,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.500877+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:20.500877+0000","last_peered":"2024-09-05T13:21:20.500877+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:20.500877+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10199040,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14513,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1954}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18679","reported_seq":18820,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.494534+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:20.494534+0000","last_peered":"2024-09-05T13:21:20.494534+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:20.494534+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9834496,"num_objects":2401,"num_object_clones":0,"num_object_copies":4802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14809,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18731","reported_seq":18877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:20.504497+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:20.504497+0000","last_peered":"2024-09-05T13:21:20.504497+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:20.504497+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10080739,"num_objects":2462,"num_object_clones":0,"num_object_copies":4924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15399,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":162628091,"num_objects":39706,"num_object_clones":0,"num_object_copies":79412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39706,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251481,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10575,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544509952,"data_stored":544429548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540760,"kb_used_data":224456,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93831080,"statfs":{"total":96636764160,"available":96083025920,"internally_reserved":0,"allocated":229842944,"data_stored":229572942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":323874477},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645124,"kb_used_data":190068,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93726716,"statfs":{"total":96636764160,"available":95976157184,"internally_reserved":0,"allocated":194629632,"data_stored":194359625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,0,9,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508772,"kb_used_data":120272,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93863068,"statfs":{"total":96636764160,"available":96115781632,"internally_reserved":0,"allocated":123158528,"data_stored":122823028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":397799087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":193269760,"data_stored":193266147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122490880,"data_stored":122421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":228749312,"data_stored":228741627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:25.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:25.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18439", 'reported_seq': 18520, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:20.445043+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:20.445043+0000', 'last_peered': '2024-09-05T13:21:20.445043+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:20.445043+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10375168, 'num_objects': 2533, 'num_object_clones': 0, 'num_object_copies': 5066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18439, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1888}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:25.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:25.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'18652", 'reported_seq': 22276, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:18.727279+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:18.727279+0000', 'last_peered': '2024-09-05T13:21:18.727279+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:18.727279+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10416152, 'num_objects': 2544, 'num_object_clones': 0, 'num_object_copies': 5088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2748, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2544, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16463, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1745, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -204}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'18672", 'reported_seq': 18816, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:20.520593+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:20.520593+0000', 'last_peered': '2024-09-05T13:21:20.520593+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:20.520593+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10170368, 'num_objects': 2483, 'num_object_clones': 0, 'num_object_copies': 4966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15127, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1968}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'18472", 'reported_seq': 18613, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:18.742613+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:18.742613+0000', 'last_peered': '2024-09-05T13:21:18.742613+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:18.742613+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10735616, 'num_objects': 2621, 'num_object_clones': 0, 'num_object_copies': 5242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16126, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1830}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'18446", 'reported_seq': 18590, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:20.491429+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:20.491429+0000', 'last_peered': '2024-09-05T13:21:20.491429+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:20.491429+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10092544, 'num_objects': 2464, 'num_object_clones': 0, 'num_object_copies': 4928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14813, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1990}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'18140", 'reported_seq': 18269, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:18.749736+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:18.749736+0000', 'last_peered': '2024-09-05T13:21:18.749736+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:18.749736+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10002432, 'num_objects': 2442, 'num_object_clones': 0, 'num_object_copies': 4884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15649, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1945}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:25.314 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'18429", 'reported_seq': 18575, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:20.456035+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:20.456035+0000', 'last_peered': '2024-09-05T13:21:20.456035+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:20.456035+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10031104, 'num_objects': 2449, 'num_object_clones': 0, 'num_object_copies': 4898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14580, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1970}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:25.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:25.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'18614", 'reported_seq': 18807, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:20.500877+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:20.500877+0000', 'last_peered': '2024-09-05T13:21:20.500877+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:20.500877+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10199040, 'num_objects': 2490, 'num_object_clones': 0, 'num_object_copies': 4980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2490, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14513, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1954}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:25.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:25.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'18679", 'reported_seq': 18820, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:20.494534+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:20.494534+0000', 'last_peered': '2024-09-05T13:21:20.494534+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:20.494534+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9834496, 'num_objects': 2401, 'num_object_clones': 0, 'num_object_copies': 4802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14809, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2165}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:25.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:25.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'18731", 'reported_seq': 18877, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:20.504497+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:20.504497+0000', 'last_peered': '2024-09-05T13:21:20.504497+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:20.504497+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10080739, 'num_objects': 2462, 'num_object_clones': 0, 'num_object_copies': 4924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15399, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2103}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:25.315 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:26.007 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:26.007 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42923/79416 objects degraded (54.048%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":612,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":39708,"data_bytes":162628110,"bytes_used":1735327744,"bytes_avail":288174964736,"bytes_total":289910292480,"degraded_objects":42923,"degraded_total":79416,"degraded_ratio":0.54048302609046039,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":183,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":385},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:26.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:26.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:26.593 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:26.594 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:26.620 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-05T13:21:25.719233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":153055758,"num_objects":37371,"num_object_clones":0,"num_object_copies":74742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253820,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10672,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1712384,"kb_used_data":516432,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281403136,"statfs":{"total":289910292480,"available":288156811264,"internally_reserved":0,"allocated":528826368,"data_stored":527950859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1224592390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,7,9,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28528640,"num_objects":-6965,"num_object_clones":0,"num_object_copies":-13930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6965,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004616"},"pg_stats":[{"pgid":"2.f","version":"47'18593","reported_seq":18674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.557181+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:25.557181+0000","last_peered":"2024-09-05T13:21:25.557181+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:25.557181+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9744384,"num_objects":2379,"num_object_clones":0,"num_object_copies":4758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18593,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2042}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18435","reported_seq":18546,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:23.789241+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:23.789241+0000","last_peered":"2024-09-05T13:21:23.789241+0000","last_clean":"2024-09-05T13:21:23.789241+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:23.789241+0000","last_undegraded":"2024-09-05T13:21:23.789241+0000","last_fullsized":"2024-09-05T13:21:23.789241+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9498624,"num_objects":2319,"num_object_clones":0,"num_object_copies":4638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16445,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18792","reported_seq":22609,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:23.769715+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:23.769715+0000","last_peered":"2024-09-05T13:21:23.769715+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:23.769715+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9842712,"num_objects":2404,"num_object_clones":0,"num_object_copies":4808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2652,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16603,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1841,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18623","reported_seq":18699,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:23.773740+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:23.773740+0000","last_peered":"2024-09-05T13:21:23.773740+0000","last_clean":"2024-09-05T13:21:23.773740+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:23.773740+0000","last_undegraded":"2024-09-05T13:21:23.773740+0000","last_fullsized":"2024-09-05T13:21:23.773740+0000","mapping_epoch":35,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9842688,"num_objects":2403,"num_object_clones":0,"num_object_copies":4806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18623,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18608","reported_seq":18748,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:25.464430+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:25.464430+0000","last_peered":"2024-09-05T13:21:25.464430+0000","last_clean":"2024-09-05T13:21:25.464430+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:25.464430+0000","last_undegraded":"2024-09-05T13:21:25.464430+0000","last_fullsized":"2024-09-05T13:21:25.464430+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318400,"num_objects":2275,"num_object_clones":0,"num_object_copies":4550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15148,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18810","reported_seq":18954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.421217+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:25.421217+0000","last_peered":"2024-09-05T13:21:25.421217+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:25.421217+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9605120,"num_objects":2345,"num_object_clones":0,"num_object_copies":4690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15265,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18367","reported_seq":27399,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:24.649347+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:24.649347+0000","last_peered":"2024-09-05T13:21:24.649347+0000","last_clean":"2024-09-05T13:21:24.649347+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:24.649347+0000","last_undegraded":"2024-09-05T13:21:24.649347+0000","last_fullsized":"2024-09-05T13:21:24.649347+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9261056,"num_objects":2261,"num_object_clones":0,"num_object_copies":4522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14686,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18794","reported_seq":18932,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:25.495036+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:25.495036+0000","last_peered":"2024-09-05T13:21:25.495036+0000","last_clean":"2024-09-05T13:21:25.495036+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:25.495036+0000","last_undegraded":"2024-09-05T13:21:25.495036+0000","last_fullsized":"2024-09-05T13:21:25.495036+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9461760,"num_objects":2310,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15404,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18611","reported_seq":18752,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:23.765034+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:23.765034+0000","last_peered":"2024-09-05T13:21:23.765034+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:23.765034+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166272,"num_objects":2482,"num_object_clones":0,"num_object_copies":4964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16265,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18605","reported_seq":18749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.541595+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:25.541595+0000","last_peered":"2024-09-05T13:21:25.541595+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:25.541595+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9441280,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14972,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2149}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18544","reported_seq":27451,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:23.807226+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:23.807226+0000","last_peered":"2024-09-05T13:21:23.807226+0000","last_clean":"2024-09-05T13:21:23.807226+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:23.807226+0000","last_undegraded":"2024-09-05T13:21:23.807226+0000","last_fullsized":"2024-09-05T13:21:23.807226+0000","mapping_epoch":44,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9695232,"num_objects":2367,"num_object_clones":0,"num_object_copies":4734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16139,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18260","reported_seq":18389,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:23.728974+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:23.728974+0000","last_peered":"2024-09-05T13:21:23.728974+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:23.728974+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9510912,"num_objects":2322,"num_object_clones":0,"num_object_copies":4644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15769,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2065}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18584","reported_seq":18730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.463804+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:25.463804+0000","last_peered":"2024-09-05T13:21:25.463804+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:25.463804+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9396224,"num_objects":2294,"num_object_clones":0,"num_object_copies":4588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14735,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18761","reported_seq":18953,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.521276+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:25.521276+0000","last_peered":"2024-09-05T13:21:25.521276+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:25.521276+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9596928,"num_objects":2343,"num_object_clones":0,"num_object_copies":4686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14660,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18827","reported_seq":18972,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.354821+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:25.354821+0000","last_peered":"2024-09-05T13:21:25.354821+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:25.354821+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14957,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18886","reported_seq":19031,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.437588+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:25.437588+0000","last_peered":"2024-09-05T13:21:25.437588+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:25.437588+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9445859,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15554,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2258}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":153055739,"num_objects":37369,"num_object_clones":0,"num_object_copies":74738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253818,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10671,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":525312000,"data_stored":525231596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558480,"kb_used_data":219968,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93813360,"statfs":{"total":96636764160,"available":96064880640,"internally_reserved":0,"allocated":225247232,"data_stored":224977230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625596,"kb_used_data":183020,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93746244,"statfs":{"total":96636764160,"available":95996153856,"internally_reserved":0,"allocated":187412480,"data_stored":187142473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528308,"kb_used_data":113444,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93843532,"statfs":{"total":96636764160,"available":96095776768,"internally_reserved":0,"allocated":116166656,"data_stored":115831156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":424799919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185958400,"data_stored":185954787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115220480,"data_stored":115151374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":224133120,"data_stored":224125435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:29.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:30.185 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:30.188 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:30.205 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-05T13:21:29.720795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149525006,"num_objects":36509,"num_object_clones":0,"num_object_copies":73018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42721,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36509,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254682,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10778,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160965,"ondisk_log_size":160965,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1712040,"kb_used_data":511928,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281403480,"statfs":{"total":289910292480,"available":288157163520,"internally_reserved":0,"allocated":524214272,"data_stored":523338763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1228852230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,4,5,9,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22720512,"num_objects":-5547,"num_object_clones":0,"num_object_copies":-11094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5547,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004779"},"pg_stats":[{"pgid":"2.f","version":"47'18593","reported_seq":18674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.557181+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:25.557181+0000","last_peered":"2024-09-05T13:21:25.557181+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:25.557181+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9744384,"num_objects":2379,"num_object_clones":0,"num_object_copies":4758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18593,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2042}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18590","reported_seq":18701,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.812423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:28.812423+0000","last_peered":"2024-09-05T13:21:28.812423+0000","last_clean":"2024-09-05T13:21:28.812423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:28.812423+0000","last_undegraded":"2024-09-05T13:21:28.812423+0000","last_fullsized":"2024-09-05T13:21:28.812423+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8863744,"num_objects":2164,"num_object_clones":0,"num_object_copies":4328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18926","reported_seq":22953,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.813786+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:28.813786+0000","last_peered":"2024-09-05T13:21:28.813786+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:28.813786+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9293848,"num_objects":2270,"num_object_clones":0,"num_object_copies":4540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16737,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1947,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18763","reported_seq":18839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.820865+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:28.820865+0000","last_peered":"2024-09-05T13:21:28.820865+0000","last_clean":"2024-09-05T13:21:28.820865+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:28.820865+0000","last_undegraded":"2024-09-05T13:21:28.820865+0000","last_fullsized":"2024-09-05T13:21:28.820865+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9269248,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18763,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18608","reported_seq":18748,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:25.464430+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:25.464430+0000","last_peered":"2024-09-05T13:21:25.464430+0000","last_clean":"2024-09-05T13:21:25.464430+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:25.464430+0000","last_undegraded":"2024-09-05T13:21:25.464430+0000","last_fullsized":"2024-09-05T13:21:25.464430+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318400,"num_objects":2275,"num_object_clones":0,"num_object_copies":4550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15148,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18810","reported_seq":18954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.421217+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:25.421217+0000","last_peered":"2024-09-05T13:21:25.421217+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:25.421217+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9605120,"num_objects":2345,"num_object_clones":0,"num_object_copies":4690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15265,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18367","reported_seq":27399,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:24.649347+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:24.649347+0000","last_peered":"2024-09-05T13:21:24.649347+0000","last_clean":"2024-09-05T13:21:24.649347+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:24.649347+0000","last_undegraded":"2024-09-05T13:21:24.649347+0000","last_fullsized":"2024-09-05T13:21:24.649347+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9261056,"num_objects":2261,"num_object_clones":0,"num_object_copies":4522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14686,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18794","reported_seq":18932,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:25.495036+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:25.495036+0000","last_peered":"2024-09-05T13:21:25.495036+0000","last_clean":"2024-09-05T13:21:25.495036+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:25.495036+0000","last_undegraded":"2024-09-05T13:21:25.495036+0000","last_fullsized":"2024-09-05T13:21:25.495036+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9461760,"num_objects":2310,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15404,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18769","reported_seq":18910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.806096+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:28.806096+0000","last_peered":"2024-09-05T13:21:28.806096+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:28.806096+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16423,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18605","reported_seq":18749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.541595+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:25.541595+0000","last_peered":"2024-09-05T13:21:25.541595+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:25.541595+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9441280,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14972,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2149}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18693","reported_seq":27600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.812347+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:28.812347+0000","last_peered":"2024-09-05T13:21:28.812347+0000","last_clean":"2024-09-05T13:21:28.812347+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:28.812347+0000","last_undegraded":"2024-09-05T13:21:28.812347+0000","last_fullsized":"2024-09-05T13:21:28.812347+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9084928,"num_objects":2218,"num_object_clones":0,"num_object_copies":4436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16288,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18386","reported_seq":18515,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.730145+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:28.730145+0000","last_peered":"2024-09-05T13:21:28.730145+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:28.730145+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8994816,"num_objects":2196,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15895,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2191}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18584","reported_seq":18730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.463804+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:25.463804+0000","last_peered":"2024-09-05T13:21:25.463804+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:25.463804+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9396224,"num_objects":2294,"num_object_clones":0,"num_object_copies":4588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14735,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18761","reported_seq":18953,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.521276+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:25.521276+0000","last_peered":"2024-09-05T13:21:25.521276+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:25.521276+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9596928,"num_objects":2343,"num_object_clones":0,"num_object_copies":4686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14660,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18827","reported_seq":18972,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.354821+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:25.354821+0000","last_peered":"2024-09-05T13:21:25.354821+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:25.354821+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14957,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18886","reported_seq":19031,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.437588+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:25.437588+0000","last_peered":"2024-09-05T13:21:25.437588+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:25.437588+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9445859,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15554,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2258}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":149524987,"num_objects":36507,"num_object_clones":0,"num_object_copies":73014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42721,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254680,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10777,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520630272,"data_stored":520549868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160962,"ondisk_log_size":160962,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558136,"kb_used_data":215464,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93813704,"statfs":{"total":96636764160,"available":96065232896,"internally_reserved":0,"allocated":220635136,"data_stored":220365134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625596,"kb_used_data":183020,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93746244,"statfs":{"total":96636764160,"available":95996153856,"internally_reserved":0,"allocated":187412480,"data_stored":187142473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528308,"kb_used_data":113444,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93843532,"statfs":{"total":96636764160,"available":96095776768,"internally_reserved":0,"allocated":116166656,"data_stored":115831156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":424799919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185958400,"data_stored":185954787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115220480,"data_stored":115151374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":219451392,"data_stored":219443707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:30.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:30.776 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:30.778 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:30.801 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-05T13:21:29.720795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":149525006,"num_objects":36509,"num_object_clones":0,"num_object_copies":73018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42721,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36509,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254682,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10778,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160965,"ondisk_log_size":160965,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1712040,"kb_used_data":511928,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281403480,"statfs":{"total":289910292480,"available":288157163520,"internally_reserved":0,"allocated":524214272,"data_stored":523338763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1228852230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,4,5,9,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22720512,"num_objects":-5547,"num_object_clones":0,"num_object_copies":-11094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5547,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004779"},"pg_stats":[{"pgid":"2.f","version":"47'18593","reported_seq":18674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.557181+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:25.557181+0000","last_peered":"2024-09-05T13:21:25.557181+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:25.557181+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9744384,"num_objects":2379,"num_object_clones":0,"num_object_copies":4758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18593,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2042}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18590","reported_seq":18701,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.812423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:28.812423+0000","last_peered":"2024-09-05T13:21:28.812423+0000","last_clean":"2024-09-05T13:21:28.812423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:28.812423+0000","last_undegraded":"2024-09-05T13:21:28.812423+0000","last_fullsized":"2024-09-05T13:21:28.812423+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8863744,"num_objects":2164,"num_object_clones":0,"num_object_copies":4328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18926","reported_seq":22953,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.813786+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:28.813786+0000","last_peered":"2024-09-05T13:21:28.813786+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:28.813786+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9293848,"num_objects":2270,"num_object_clones":0,"num_object_copies":4540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16737,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1947,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18763","reported_seq":18839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.820865+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:28.820865+0000","last_peered":"2024-09-05T13:21:28.820865+0000","last_clean":"2024-09-05T13:21:28.820865+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:28.820865+0000","last_undegraded":"2024-09-05T13:21:28.820865+0000","last_fullsized":"2024-09-05T13:21:28.820865+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9269248,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18763,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18608","reported_seq":18748,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:25.464430+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:25.464430+0000","last_peered":"2024-09-05T13:21:25.464430+0000","last_clean":"2024-09-05T13:21:25.464430+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:25.464430+0000","last_undegraded":"2024-09-05T13:21:25.464430+0000","last_fullsized":"2024-09-05T13:21:25.464430+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9318400,"num_objects":2275,"num_object_clones":0,"num_object_copies":4550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15148,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18810","reported_seq":18954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.421217+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:25.421217+0000","last_peered":"2024-09-05T13:21:25.421217+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:25.421217+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9605120,"num_objects":2345,"num_object_clones":0,"num_object_copies":4690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15265,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18367","reported_seq":27399,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:24.649347+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:24.649347+0000","last_peered":"2024-09-05T13:21:24.649347+0000","last_clean":"2024-09-05T13:21:24.649347+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:24.649347+0000","last_undegraded":"2024-09-05T13:21:24.649347+0000","last_fullsized":"2024-09-05T13:21:24.649347+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9261056,"num_objects":2261,"num_object_clones":0,"num_object_copies":4522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14686,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18794","reported_seq":18932,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:25.495036+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:25.495036+0000","last_peered":"2024-09-05T13:21:25.495036+0000","last_clean":"2024-09-05T13:21:25.495036+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:25.495036+0000","last_undegraded":"2024-09-05T13:21:25.495036+0000","last_fullsized":"2024-09-05T13:21:25.495036+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9461760,"num_objects":2310,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15404,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18769","reported_seq":18910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.806096+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:28.806096+0000","last_peered":"2024-09-05T13:21:28.806096+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:28.806096+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16423,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18605","reported_seq":18749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.541595+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:25.541595+0000","last_peered":"2024-09-05T13:21:25.541595+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:25.541595+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9441280,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14972,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2149}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18693","reported_seq":27600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.812347+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:28.812347+0000","last_peered":"2024-09-05T13:21:28.812347+0000","last_clean":"2024-09-05T13:21:28.812347+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:28.812347+0000","last_undegraded":"2024-09-05T13:21:28.812347+0000","last_fullsized":"2024-09-05T13:21:28.812347+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9084928,"num_objects":2218,"num_object_clones":0,"num_object_copies":4436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16288,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18386","reported_seq":18515,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.730145+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:28.730145+0000","last_peered":"2024-09-05T13:21:28.730145+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:28.730145+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8994816,"num_objects":2196,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15895,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2191}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18584","reported_seq":18730,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.463804+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:25.463804+0000","last_peered":"2024-09-05T13:21:25.463804+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:25.463804+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9396224,"num_objects":2294,"num_object_clones":0,"num_object_copies":4588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14735,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18761","reported_seq":18953,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.521276+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:25.521276+0000","last_peered":"2024-09-05T13:21:25.521276+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:25.521276+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9596928,"num_objects":2343,"num_object_clones":0,"num_object_copies":4686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14660,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2101}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18827","reported_seq":18972,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.354821+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:25.354821+0000","last_peered":"2024-09-05T13:21:25.354821+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:25.354821+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14957,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18886","reported_seq":19031,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:25.437588+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:25.437588+0000","last_peered":"2024-09-05T13:21:25.437588+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:25.437588+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9445859,"num_objects":2307,"num_object_clones":0,"num_object_copies":4614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15554,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2258}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":149524987,"num_objects":36507,"num_object_clones":0,"num_object_copies":73014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42721,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254680,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10777,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520630272,"data_stored":520549868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160962,"ondisk_log_size":160962,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558136,"kb_used_data":215464,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93813704,"statfs":{"total":96636764160,"available":96065232896,"internally_reserved":0,"allocated":220635136,"data_stored":220365134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625596,"kb_used_data":183020,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93746244,"statfs":{"total":96636764160,"available":95996153856,"internally_reserved":0,"allocated":187412480,"data_stored":187142473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528308,"kb_used_data":113444,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93843532,"statfs":{"total":96636764160,"available":96095776768,"internally_reserved":0,"allocated":116166656,"data_stored":115831156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":424799919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185958400,"data_stored":185954787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115220480,"data_stored":115151374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":219451392,"data_stored":219443707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:30.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:30.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18593", 'reported_seq': 18674, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:25.557181+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:25.557181+0000', 'last_peered': '2024-09-05T13:21:25.557181+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:25.557181+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9744384, 'num_objects': 2379, 'num_object_clones': 0, 'num_object_copies': 4758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18593, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2042}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'18926", 'reported_seq': 22953, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:28.813786+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:28.813786+0000', 'last_peered': '2024-09-05T13:21:28.813786+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:28.813786+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9293848, 'num_objects': 2270, 'num_object_clones': 0, 'num_object_copies': 4540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2546, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2270, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16737, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1947, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'18810", 'reported_seq': 18954, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:25.421217+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:25.421217+0000', 'last_peered': '2024-09-05T13:21:25.421217+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:25.421217+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9605120, 'num_objects': 2345, 'num_object_clones': 0, 'num_object_copies': 4690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2345, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15265, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2106}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'18769", 'reported_seq': 18910, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:28.806096+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:28.806096+0000', 'last_peered': '2024-09-05T13:21:28.806096+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:28.806096+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9519104, 'num_objects': 2324, 'num_object_clones': 0, 'num_object_copies': 4648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16423, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2127}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'18605", 'reported_seq': 18749, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:25.541595+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:25.541595+0000', 'last_peered': '2024-09-05T13:21:25.541595+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:25.541595+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9441280, 'num_objects': 2305, 'num_object_clones': 0, 'num_object_copies': 4610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14972, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2149}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:30.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'18386", 'reported_seq': 18515, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:28.730145+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:28.730145+0000', 'last_peered': '2024-09-05T13:21:28.730145+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:28.730145+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8994816, 'num_objects': 2196, 'num_object_clones': 0, 'num_object_copies': 4392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2196, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15895, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2191}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:30.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:30.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'18584", 'reported_seq': 18730, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:25.463804+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:25.463804+0000', 'last_peered': '2024-09-05T13:21:25.463804+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:25.463804+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9396224, 'num_objects': 2294, 'num_object_clones': 0, 'num_object_copies': 4588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2294, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14735, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:30.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:30.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'18761", 'reported_seq': 18953, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:25.521276+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:25.521276+0000', 'last_peered': '2024-09-05T13:21:25.521276+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:25.521276+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9596928, 'num_objects': 2343, 'num_object_clones': 0, 'num_object_copies': 4686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14660, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:30.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:30.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'18827", 'reported_seq': 18972, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:25.354821+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:25.354821+0000', 'last_peered': '2024-09-05T13:21:25.354821+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:25.354821+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9228288, 'num_objects': 2253, 'num_object_clones': 0, 'num_object_copies': 4506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14957, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:30.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:30.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'18886", 'reported_seq': 19031, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:25.437588+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:25.437588+0000', 'last_peered': '2024-09-05T13:21:25.437588+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:25.437588+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9445859, 'num_objects': 2307, 'num_object_clones': 0, 'num_object_copies': 4614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15554, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2258}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:30.805 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:31.498 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:31.499 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42721/73018 objects degraded (58.507%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":618,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":36509,"data_bytes":149525006,"bytes_used":1753128960,"bytes_avail":288157163520,"bytes_total":289910292480,"degraded_objects":42721,"degraded_total":73018,"degraded_ratio":0.58507491303514203,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":462},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:31.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:31.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:32.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:32.109 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:32.128 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-05T13:21:31.721530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":143553038,"num_objects":35051,"num_object_clones":0,"num_object_copies":70102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42721,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35051,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256140,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10778,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1728672,"kb_used_data":498012,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281386848,"statfs":{"total":289910292480,"available":288140132352,"internally_reserved":0,"allocated":509964288,"data_stored":509088779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1260112902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,3,6,10,14,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24535040,"num_objects":-5990,"num_object_clones":0,"num_object_copies":-11980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5990,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004754"},"pg_stats":[{"pgid":"2.f","version":"47'18728","reported_seq":18809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.503376+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:30.503376+0000","last_peered":"2024-09-05T13:21:30.503376+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:30.503376+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9191424,"num_objects":2244,"num_object_clones":0,"num_object_copies":4488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18728,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2177}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18590","reported_seq":18701,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.812423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:28.812423+0000","last_peered":"2024-09-05T13:21:28.812423+0000","last_clean":"2024-09-05T13:21:28.812423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:28.812423+0000","last_undegraded":"2024-09-05T13:21:28.812423+0000","last_fullsized":"2024-09-05T13:21:28.812423+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8863744,"num_objects":2164,"num_object_clones":0,"num_object_copies":4328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18926","reported_seq":22953,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.813786+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:28.813786+0000","last_peered":"2024-09-05T13:21:28.813786+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:28.813786+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9293848,"num_objects":2270,"num_object_clones":0,"num_object_copies":4540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16737,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1947,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18763","reported_seq":18839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.820865+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:28.820865+0000","last_peered":"2024-09-05T13:21:28.820865+0000","last_clean":"2024-09-05T13:21:28.820865+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:28.820865+0000","last_undegraded":"2024-09-05T13:21:28.820865+0000","last_fullsized":"2024-09-05T13:21:28.820865+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9269248,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18763,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18747","reported_seq":18887,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:30.470509+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:30.470509+0000","last_peered":"2024-09-05T13:21:30.470509+0000","last_clean":"2024-09-05T13:21:30.470509+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:30.470509+0000","last_undegraded":"2024-09-05T13:21:30.470509+0000","last_fullsized":"2024-09-05T13:21:30.470509+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8749056,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15287,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18981","reported_seq":19125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.516670+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:30.516670+0000","last_peered":"2024-09-05T13:21:30.516670+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:30.516670+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8904704,"num_objects":2174,"num_object_clones":0,"num_object_copies":4348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15436,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18517","reported_seq":27549,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:29.766594+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:29.766594+0000","last_peered":"2024-09-05T13:21:29.766594+0000","last_clean":"2024-09-05T13:21:29.766594+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:29.766594+0000","last_undegraded":"2024-09-05T13:21:29.766594+0000","last_fullsized":"2024-09-05T13:21:29.766594+0000","mapping_epoch":46,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8646656,"num_objects":2111,"num_object_clones":0,"num_object_copies":4222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14836,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18952","reported_seq":19090,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:30.582390+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:30.582390+0000","last_peered":"2024-09-05T13:21:30.582390+0000","last_clean":"2024-09-05T13:21:30.582390+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:30.582390+0000","last_undegraded":"2024-09-05T13:21:30.582390+0000","last_fullsized":"2024-09-05T13:21:30.582390+0000","mapping_epoch":34,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8814592,"num_objects":2152,"num_object_clones":0,"num_object_copies":4304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15562,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18769","reported_seq":18910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.806096+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:28.806096+0000","last_peered":"2024-09-05T13:21:28.806096+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:28.806096+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16423,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18745","reported_seq":18889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.582661+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:30.582661+0000","last_peered":"2024-09-05T13:21:30.582661+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:30.582661+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15112,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2289}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18693","reported_seq":27600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.812347+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:28.812347+0000","last_peered":"2024-09-05T13:21:28.812347+0000","last_clean":"2024-09-05T13:21:28.812347+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:28.812347+0000","last_undegraded":"2024-09-05T13:21:28.812347+0000","last_fullsized":"2024-09-05T13:21:28.812347+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9084928,"num_objects":2218,"num_object_clones":0,"num_object_copies":4436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16288,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18386","reported_seq":18515,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.730145+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:28.730145+0000","last_peered":"2024-09-05T13:21:28.730145+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:28.730145+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8994816,"num_objects":2196,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15895,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2191}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18725","reported_seq":18871,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.571671+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:30.571671+0000","last_peered":"2024-09-05T13:21:30.571671+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:30.571671+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8818688,"num_objects":2153,"num_object_clones":0,"num_object_copies":4306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14876,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18901","reported_seq":19094,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.550384+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:30.550384+0000","last_peered":"2024-09-05T13:21:30.550384+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:30.550384+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9023488,"num_objects":2203,"num_object_clones":0,"num_object_copies":4406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14800,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18952","reported_seq":19095,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.389775+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:30.389775+0000","last_peered":"2024-09-05T13:21:30.389775+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:30.389775+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15082,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2438}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19045","reported_seq":19191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.500112+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:30.500112+0000","last_peered":"2024-09-05T13:21:30.500112+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:30.500112+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8794595,"num_objects":2148,"num_object_clones":0,"num_object_copies":4296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":143553019,"num_objects":35049,"num_object_clones":0,"num_object_copies":70098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42721,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256138,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10777,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":506159104,"data_stored":506078700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558136,"kb_used_data":215464,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93813704,"statfs":{"total":96636764160,"available":96065232896,"internally_reserved":0,"allocated":220635136,"data_stored":220365134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626976,"kb_used_data":176080,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93744864,"statfs":{"total":96636764160,"available":95994740736,"internally_reserved":0,"allocated":180305920,"data_stored":180035913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543560,"kb_used_data":106468,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93828280,"statfs":{"total":96636764160,"available":96080158720,"internally_reserved":0,"allocated":109023232,"data_stored":108687732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":447540911},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,2,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178700288,"data_stored":178696675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":108007424,"data_stored":107938318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":219451392,"data_stored":219443707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:35.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:35.680 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:35.681 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:35.704 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-05T13:21:33.722188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":143553038,"num_objects":35051,"num_object_clones":0,"num_object_copies":70102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42721,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35051,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256140,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10778,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1728672,"kb_used_data":498012,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281386848,"statfs":{"total":289910292480,"available":288140132352,"internally_reserved":0,"allocated":509964288,"data_stored":509088779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1260112902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,3,6,10,14,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19075072,"num_objects":-4657,"num_object_clones":0,"num_object_copies":-9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-202,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004607"},"pg_stats":[{"pgid":"2.f","version":"47'18728","reported_seq":18809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.503376+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:30.503376+0000","last_peered":"2024-09-05T13:21:30.503376+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:30.503376+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9191424,"num_objects":2244,"num_object_clones":0,"num_object_copies":4488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18728,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2177}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18590","reported_seq":18701,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.812423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:28.812423+0000","last_peered":"2024-09-05T13:21:28.812423+0000","last_clean":"2024-09-05T13:21:28.812423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:28.812423+0000","last_undegraded":"2024-09-05T13:21:28.812423+0000","last_fullsized":"2024-09-05T13:21:28.812423+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8863744,"num_objects":2164,"num_object_clones":0,"num_object_copies":4328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18926","reported_seq":22953,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.813786+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:28.813786+0000","last_peered":"2024-09-05T13:21:28.813786+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:28.813786+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9293848,"num_objects":2270,"num_object_clones":0,"num_object_copies":4540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2546,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16737,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1947,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18763","reported_seq":18839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.820865+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:28.820865+0000","last_peered":"2024-09-05T13:21:28.820865+0000","last_clean":"2024-09-05T13:21:28.820865+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:28.820865+0000","last_undegraded":"2024-09-05T13:21:28.820865+0000","last_fullsized":"2024-09-05T13:21:28.820865+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9269248,"num_objects":2263,"num_object_clones":0,"num_object_copies":4526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18763,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18747","reported_seq":18887,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:30.470509+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:30.470509+0000","last_peered":"2024-09-05T13:21:30.470509+0000","last_clean":"2024-09-05T13:21:30.470509+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:30.470509+0000","last_undegraded":"2024-09-05T13:21:30.470509+0000","last_fullsized":"2024-09-05T13:21:30.470509+0000","mapping_epoch":34,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8749056,"num_objects":2136,"num_object_clones":0,"num_object_copies":4272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15287,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18981","reported_seq":19125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.516670+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:30.516670+0000","last_peered":"2024-09-05T13:21:30.516670+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:30.516670+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8904704,"num_objects":2174,"num_object_clones":0,"num_object_copies":4348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15436,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18517","reported_seq":27549,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:29.766594+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:29.766594+0000","last_peered":"2024-09-05T13:21:29.766594+0000","last_clean":"2024-09-05T13:21:29.766594+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:29.766594+0000","last_undegraded":"2024-09-05T13:21:29.766594+0000","last_fullsized":"2024-09-05T13:21:29.766594+0000","mapping_epoch":46,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8646656,"num_objects":2111,"num_object_clones":0,"num_object_copies":4222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14836,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'18952","reported_seq":19090,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:30.582390+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:30.582390+0000","last_peered":"2024-09-05T13:21:30.582390+0000","last_clean":"2024-09-05T13:21:30.582390+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:30.582390+0000","last_undegraded":"2024-09-05T13:21:30.582390+0000","last_fullsized":"2024-09-05T13:21:30.582390+0000","mapping_epoch":34,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8814592,"num_objects":2152,"num_object_clones":0,"num_object_copies":4304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15562,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18769","reported_seq":18910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.806096+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:28.806096+0000","last_peered":"2024-09-05T13:21:28.806096+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:28.806096+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16423,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18745","reported_seq":18889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.582661+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:30.582661+0000","last_peered":"2024-09-05T13:21:30.582661+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:30.582661+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15112,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2289}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18693","reported_seq":27600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:28.812347+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:28.812347+0000","last_peered":"2024-09-05T13:21:28.812347+0000","last_clean":"2024-09-05T13:21:28.812347+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:28.812347+0000","last_undegraded":"2024-09-05T13:21:28.812347+0000","last_fullsized":"2024-09-05T13:21:28.812347+0000","mapping_epoch":44,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9084928,"num_objects":2218,"num_object_clones":0,"num_object_copies":4436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16288,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18386","reported_seq":18515,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:28.730145+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:28.730145+0000","last_peered":"2024-09-05T13:21:28.730145+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:28.730145+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8994816,"num_objects":2196,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15895,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2191}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18725","reported_seq":18871,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.571671+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:30.571671+0000","last_peered":"2024-09-05T13:21:30.571671+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:30.571671+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8818688,"num_objects":2153,"num_object_clones":0,"num_object_copies":4306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14876,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18901","reported_seq":19094,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.550384+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:30.550384+0000","last_peered":"2024-09-05T13:21:30.550384+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:30.550384+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9023488,"num_objects":2203,"num_object_clones":0,"num_object_copies":4406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14800,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2241}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18952","reported_seq":19095,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.389775+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:30.389775+0000","last_peered":"2024-09-05T13:21:30.389775+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:30.389775+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8716288,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15082,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2438}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19045","reported_seq":19191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:30.500112+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:30.500112+0000","last_peered":"2024-09-05T13:21:30.500112+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:30.500112+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8794595,"num_objects":2148,"num_object_clones":0,"num_object_copies":4296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":143553019,"num_objects":35049,"num_object_clones":0,"num_object_copies":70098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42721,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":256138,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10777,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":506159104,"data_stored":506078700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558136,"kb_used_data":215464,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93813704,"statfs":{"total":96636764160,"available":96065232896,"internally_reserved":0,"allocated":220635136,"data_stored":220365134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626976,"kb_used_data":176080,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93744864,"statfs":{"total":96636764160,"available":95994740736,"internally_reserved":0,"allocated":180305920,"data_stored":180035913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,2,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543560,"kb_used_data":106468,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93828280,"statfs":{"total":96636764160,"available":96080158720,"internally_reserved":0,"allocated":109023232,"data_stored":108687732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":447540911},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,2,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178700288,"data_stored":178696675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":108007424,"data_stored":107938318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":219451392,"data_stored":219443707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:35.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:36.244 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:36.269 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-05T13:21:35.722750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134005262,"num_objects":32720,"num_object_clones":0,"num_object_copies":65440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258471,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10880,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1728376,"kb_used_data":479684,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281387144,"statfs":{"total":289910292480,"available":288140435456,"internally_reserved":0,"allocated":491196416,"data_stored":490320907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1278594054},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,7,17,11,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28622848,"num_objects":-6988,"num_object_clones":0,"num_object_copies":-13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004361"},"pg_stats":[{"pgid":"2.f","version":"47'18858","reported_seq":18939,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.598128+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:35.598128+0000","last_peered":"2024-09-05T13:21:35.598128+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:35.598128+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8658944,"num_objects":2114,"num_object_clones":0,"num_object_copies":4228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18858,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18716","reported_seq":18827,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:33.834153+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:33.834153+0000","last_peered":"2024-09-05T13:21:33.834153+0000","last_clean":"2024-09-05T13:21:33.834153+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:33.834153+0000","last_undegraded":"2024-09-05T13:21:33.834153+0000","last_fullsized":"2024-09-05T13:21:33.834153+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8347648,"num_objects":2038,"num_object_clones":0,"num_object_copies":4076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16726,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19065","reported_seq":23299,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:33.817999+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:33.817999+0000","last_peered":"2024-09-05T13:21:33.817999+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:33.817999+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8724504,"num_objects":2131,"num_object_clones":0,"num_object_copies":4262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16876,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2049,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-313}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18903","reported_seq":18979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:33.757538+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:33.757538+0000","last_peered":"2024-09-05T13:21:33.757538+0000","last_clean":"2024-09-05T13:21:33.757538+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:33.757538+0000","last_undegraded":"2024-09-05T13:21:33.757538+0000","last_fullsized":"2024-09-05T13:21:33.757538+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8695808,"num_objects":2123,"num_object_clones":0,"num_object_copies":4246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18903,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18895","reported_seq":19035,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:35.541835+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:35.541835+0000","last_peered":"2024-09-05T13:21:35.541835+0000","last_clean":"2024-09-05T13:21:35.541835+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:35.541835+0000","last_undegraded":"2024-09-05T13:21:35.541835+0000","last_fullsized":"2024-09-05T13:21:35.541835+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8142848,"num_objects":1988,"num_object_clones":0,"num_object_copies":3976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15435,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19132","reported_seq":19276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.517365+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:35.517365+0000","last_peered":"2024-09-05T13:21:35.517365+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:35.517365+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8286208,"num_objects":2023,"num_object_clones":0,"num_object_copies":4046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15587,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18661","reported_seq":27692,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:34.824750+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:34.824750+0000","last_peered":"2024-09-05T13:21:34.824750+0000","last_clean":"2024-09-05T13:21:34.824750+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:34.824750+0000","last_undegraded":"2024-09-05T13:21:34.824750+0000","last_fullsized":"2024-09-05T13:21:34.824750+0000","mapping_epoch":46,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19111","reported_seq":19248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:35.555828+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:35.555828+0000","last_peered":"2024-09-05T13:21:35.555828+0000","last_clean":"2024-09-05T13:21:35.555828+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:35.555828+0000","last_undegraded":"2024-09-05T13:21:35.555828+0000","last_fullsized":"2024-09-05T13:21:35.555828+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15721,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18928","reported_seq":19069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:33.824115+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:33.824115+0000","last_peered":"2024-09-05T13:21:33.824115+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:33.824115+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16582,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2286}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18891","reported_seq":19035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.517743+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:35.517743+0000","last_peered":"2024-09-05T13:21:35.517743+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:35.517743+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8269824,"num_objects":2019,"num_object_clones":0,"num_object_copies":4038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15258,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18829","reported_seq":27736,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:33.761631+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:33.761631+0000","last_peered":"2024-09-05T13:21:33.761631+0000","last_clean":"2024-09-05T13:21:33.761631+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:33.761631+0000","last_undegraded":"2024-09-05T13:21:33.761631+0000","last_fullsized":"2024-09-05T13:21:33.761631+0000","mapping_epoch":44,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16424,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18534","reported_seq":18663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:33.784304+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:33.784304+0000","last_peered":"2024-09-05T13:21:33.784304+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:33.784304+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16043,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18872","reported_seq":19018,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.580184+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:35.580184+0000","last_peered":"2024-09-05T13:21:35.580184+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:35.580184+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8216576,"num_objects":2006,"num_object_clones":0,"num_object_copies":4012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15023,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19059","reported_seq":19252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.590401+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:35.590401+0000","last_peered":"2024-09-05T13:21:35.590401+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:35.590401+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376320,"num_objects":2045,"num_object_clones":0,"num_object_copies":4090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19088","reported_seq":19232,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.552250+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:35.552250+0000","last_peered":"2024-09-05T13:21:35.552250+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:35.552250+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8159232,"num_objects":1992,"num_object_clones":0,"num_object_copies":3984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15218,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19209","reported_seq":19355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.567836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:35.567836+0000","last_peered":"2024-09-05T13:21:35.567836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:35.567836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8122851,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15877,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":134005243,"num_objects":32718,"num_object_clones":0,"num_object_copies":65436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258469,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10879,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":487133184,"data_stored":487052780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557632,"kb_used_data":210796,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93814208,"statfs":{"total":96636764160,"available":96065748992,"internally_reserved":0,"allocated":215855104,"data_stored":215585102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624432,"kb_used_data":169376,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93747408,"statfs":{"total":96636764160,"available":95997345792,"internally_reserved":0,"allocated":173441024,"data_stored":173171017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546312,"kb_used_data":99512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93825528,"statfs":{"total":96636764160,"available":96077340672,"internally_reserved":0,"allocated":101900288,"data_stored":101564788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171565056,"data_stored":171561443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100794368,"data_stored":100725262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":214773760,"data_stored":214766075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:36.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:36.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18858", 'reported_seq': 18939, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:35.598128+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:35.598128+0000', 'last_peered': '2024-09-05T13:21:35.598128+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:35.598128+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8658944, 'num_objects': 2114, 'num_object_clones': 0, 'num_object_copies': 4228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2114, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18858, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2307}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:36.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:36.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'19065", 'reported_seq': 23299, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:33.817999+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:33.817999+0000', 'last_peered': '2024-09-05T13:21:33.817999+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:33.817999+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8724504, 'num_objects': 2131, 'num_object_clones': 0, 'num_object_copies': 4262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2444, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2131, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16876, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2049, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:36.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:36.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'19132", 'reported_seq': 19276, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:35.517365+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:35.517365+0000', 'last_peered': '2024-09-05T13:21:35.517365+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:35.517365+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'9100", 'ondisk_log_start': "43'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8286208, 'num_objects': 2023, 'num_object_clones': 0, 'num_object_copies': 4046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2023, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15587, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'18928", 'reported_seq': 19069, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:33.824115+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:33.824115+0000', 'last_peered': '2024-09-05T13:21:33.824115+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:33.824115+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8867840, 'num_objects': 2165, 'num_object_clones': 0, 'num_object_copies': 4330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16582, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2286}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'18891", 'reported_seq': 19035, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:35.517743+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:35.517743+0000', 'last_peered': '2024-09-05T13:21:35.517743+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:35.517743+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8269824, 'num_objects': 2019, 'num_object_clones': 0, 'num_object_copies': 4038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2019, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15258, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2435}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'18534", 'reported_seq': 18663, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:33.784304+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:33.784304+0000', 'last_peered': '2024-09-05T13:21:33.784304+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:33.784304+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8388608, 'num_objects': 2048, 'num_object_clones': 0, 'num_object_copies': 4096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2048, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16043, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2339}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'18872", 'reported_seq': 19018, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:35.580184+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:35.580184+0000', 'last_peered': '2024-09-05T13:21:35.580184+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:35.580184+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8216576, 'num_objects': 2006, 'num_object_clones': 0, 'num_object_copies': 4012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15023, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2413}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:36.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'19059", 'reported_seq': 19252, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:35.590401+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:35.590401+0000', 'last_peered': '2024-09-05T13:21:35.590401+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:35.590401+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8376320, 'num_objects': 2045, 'num_object_clones': 0, 'num_object_copies': 4090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14958, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:36.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:36.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'19088", 'reported_seq': 19232, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:35.552250+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:35.552250+0000', 'last_peered': '2024-09-05T13:21:35.552250+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:35.552250+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8159232, 'num_objects': 1992, 'num_object_clones': 0, 'num_object_copies': 3984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1992, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15218, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:36.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:36.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'19209", 'reported_seq': 19355, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:35.567836+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:35.567836+0000', 'last_peered': '2024-09-05T13:21:35.567836+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:35.567836+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8122851, 'num_objects': 1984, 'num_object_clones': 0, 'num_object_copies': 3968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15877, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2581}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:36.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:36.958 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:36.958 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42619/65440 objects degraded (65.127%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":623,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":32720,"data_bytes":134005262,"bytes_used":1769857024,"bytes_avail":288140435456,"bytes_total":289910292480,"degraded_objects":42619,"degraded_total":65440,"degraded_ratio":0.65126833740831291,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":582},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:36.983 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:36.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:37.572 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:37.572 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:37.600 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-05T13:21:35.722750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":134005262,"num_objects":32720,"num_object_clones":0,"num_object_copies":65440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258471,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10880,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1728376,"kb_used_data":479684,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281387144,"statfs":{"total":289910292480,"available":288140435456,"internally_reserved":0,"allocated":491196416,"data_stored":490320907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1278594054},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,7,17,11,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28622848,"num_objects":-6988,"num_object_clones":0,"num_object_copies":-13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004361"},"pg_stats":[{"pgid":"2.f","version":"47'18858","reported_seq":18939,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.598128+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:35.598128+0000","last_peered":"2024-09-05T13:21:35.598128+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:35.598128+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8658944,"num_objects":2114,"num_object_clones":0,"num_object_copies":4228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18858,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18716","reported_seq":18827,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:33.834153+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:33.834153+0000","last_peered":"2024-09-05T13:21:33.834153+0000","last_clean":"2024-09-05T13:21:33.834153+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:33.834153+0000","last_undegraded":"2024-09-05T13:21:33.834153+0000","last_fullsized":"2024-09-05T13:21:33.834153+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8347648,"num_objects":2038,"num_object_clones":0,"num_object_copies":4076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16726,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19065","reported_seq":23299,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:33.817999+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:33.817999+0000","last_peered":"2024-09-05T13:21:33.817999+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:33.817999+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8724504,"num_objects":2131,"num_object_clones":0,"num_object_copies":4262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2444,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16876,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2049,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-313}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18903","reported_seq":18979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:33.757538+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:33.757538+0000","last_peered":"2024-09-05T13:21:33.757538+0000","last_clean":"2024-09-05T13:21:33.757538+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:33.757538+0000","last_undegraded":"2024-09-05T13:21:33.757538+0000","last_fullsized":"2024-09-05T13:21:33.757538+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8695808,"num_objects":2123,"num_object_clones":0,"num_object_copies":4246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18903,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18895","reported_seq":19035,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:35.541835+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:35.541835+0000","last_peered":"2024-09-05T13:21:35.541835+0000","last_clean":"2024-09-05T13:21:35.541835+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:35.541835+0000","last_undegraded":"2024-09-05T13:21:35.541835+0000","last_fullsized":"2024-09-05T13:21:35.541835+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8142848,"num_objects":1988,"num_object_clones":0,"num_object_copies":3976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15435,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19132","reported_seq":19276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.517365+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:35.517365+0000","last_peered":"2024-09-05T13:21:35.517365+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:35.517365+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8286208,"num_objects":2023,"num_object_clones":0,"num_object_copies":4046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15587,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18661","reported_seq":27692,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:34.824750+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:34.824750+0000","last_peered":"2024-09-05T13:21:34.824750+0000","last_clean":"2024-09-05T13:21:34.824750+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:34.824750+0000","last_undegraded":"2024-09-05T13:21:34.824750+0000","last_fullsized":"2024-09-05T13:21:34.824750+0000","mapping_epoch":46,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19111","reported_seq":19248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:35.555828+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:35.555828+0000","last_peered":"2024-09-05T13:21:35.555828+0000","last_clean":"2024-09-05T13:21:35.555828+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:35.555828+0000","last_undegraded":"2024-09-05T13:21:35.555828+0000","last_fullsized":"2024-09-05T13:21:35.555828+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15721,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18928","reported_seq":19069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:33.824115+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:33.824115+0000","last_peered":"2024-09-05T13:21:33.824115+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:33.824115+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16582,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2286}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18891","reported_seq":19035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.517743+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:35.517743+0000","last_peered":"2024-09-05T13:21:35.517743+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:35.517743+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8269824,"num_objects":2019,"num_object_clones":0,"num_object_copies":4038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15258,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18829","reported_seq":27736,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:33.761631+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:33.761631+0000","last_peered":"2024-09-05T13:21:33.761631+0000","last_clean":"2024-09-05T13:21:33.761631+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:33.761631+0000","last_undegraded":"2024-09-05T13:21:33.761631+0000","last_fullsized":"2024-09-05T13:21:33.761631+0000","mapping_epoch":44,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16424,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18534","reported_seq":18663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:33.784304+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:33.784304+0000","last_peered":"2024-09-05T13:21:33.784304+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:33.784304+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2048,"num_object_clones":0,"num_object_copies":4096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16043,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18872","reported_seq":19018,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.580184+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:35.580184+0000","last_peered":"2024-09-05T13:21:35.580184+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:35.580184+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8216576,"num_objects":2006,"num_object_clones":0,"num_object_copies":4012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15023,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19059","reported_seq":19252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.590401+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:35.590401+0000","last_peered":"2024-09-05T13:21:35.590401+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:35.590401+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376320,"num_objects":2045,"num_object_clones":0,"num_object_copies":4090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19088","reported_seq":19232,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.552250+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:35.552250+0000","last_peered":"2024-09-05T13:21:35.552250+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:35.552250+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8159232,"num_objects":1992,"num_object_clones":0,"num_object_copies":3984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15218,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19209","reported_seq":19355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.567836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:35.567836+0000","last_peered":"2024-09-05T13:21:35.567836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:35.567836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8122851,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15877,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":134005243,"num_objects":32718,"num_object_clones":0,"num_object_copies":65436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32718,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258469,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10879,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":487133184,"data_stored":487052780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557632,"kb_used_data":210796,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93814208,"statfs":{"total":96636764160,"available":96065748992,"internally_reserved":0,"allocated":215855104,"data_stored":215585102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624432,"kb_used_data":169376,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93747408,"statfs":{"total":96636764160,"available":95997345792,"internally_reserved":0,"allocated":173441024,"data_stored":173171017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546312,"kb_used_data":99512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93825528,"statfs":{"total":96636764160,"available":96077340672,"internally_reserved":0,"allocated":101900288,"data_stored":101564788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171565056,"data_stored":171561443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100794368,"data_stored":100725262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":214773760,"data_stored":214766075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:40.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:41.191 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:41.193 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:41.226 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-09-05T13:21:39.724224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":130380302,"num_objects":31835,"num_object_clones":0,"num_object_copies":63670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31835,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259356,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10984,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160939,"ondisk_log_size":160939,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1711380,"kb_used_data":475172,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281404140,"statfs":{"total":289910292480,"available":288157839360,"internally_reserved":0,"allocated":486576128,"data_stored":485700619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1265814534},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,3,5,16,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22675456,"num_objects":-5536,"num_object_clones":0,"num_object_copies":-11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004190"},"pg_stats":[{"pgid":"2.f","version":"47'18858","reported_seq":18939,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.598128+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:35.598128+0000","last_peered":"2024-09-05T13:21:35.598128+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:35.598128+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8658944,"num_objects":2114,"num_object_clones":0,"num_object_copies":4228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18858,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2307}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18836","reported_seq":18946,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.820733+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:38.820733+0000","last_peered":"2024-09-05T13:21:38.820733+0000","last_clean":"2024-09-05T13:21:38.820733+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:38.820733+0000","last_undegraded":"2024-09-05T13:21:38.820733+0000","last_fullsized":"2024-09-05T13:21:38.820733+0000","mapping_epoch":35,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7856128,"num_objects":1918,"num_object_clones":0,"num_object_copies":3836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16846,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19228","reported_seq":23670,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.848069+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:38.848069+0000","last_peered":"2024-09-05T13:21:38.848069+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:38.848069+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056856,"num_objects":1968,"num_object_clones":0,"num_object_copies":3936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17039,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19055","reported_seq":19130,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.849314+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:38.849314+0000","last_peered":"2024-09-05T13:21:38.849314+0000","last_clean":"2024-09-05T13:21:38.849314+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:38.849314+0000","last_undegraded":"2024-09-05T13:21:38.849314+0000","last_fullsized":"2024-09-05T13:21:38.849314+0000","mapping_epoch":35,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8073216,"num_objects":1971,"num_object_clones":0,"num_object_copies":3942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19055,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18895","reported_seq":19035,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:35.541835+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:35.541835+0000","last_peered":"2024-09-05T13:21:35.541835+0000","last_clean":"2024-09-05T13:21:35.541835+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:35.541835+0000","last_undegraded":"2024-09-05T13:21:35.541835+0000","last_fullsized":"2024-09-05T13:21:35.541835+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8142848,"num_objects":1988,"num_object_clones":0,"num_object_copies":3976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15435,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19132","reported_seq":19276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.517365+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:35.517365+0000","last_peered":"2024-09-05T13:21:35.517365+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:35.517365+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8286208,"num_objects":2023,"num_object_clones":0,"num_object_copies":4046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15587,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18661","reported_seq":27692,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:34.824750+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:34.824750+0000","last_peered":"2024-09-05T13:21:34.824750+0000","last_clean":"2024-09-05T13:21:34.824750+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:34.824750+0000","last_undegraded":"2024-09-05T13:21:34.824750+0000","last_fullsized":"2024-09-05T13:21:34.824750+0000","mapping_epoch":46,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056832,"num_objects":1967,"num_object_clones":0,"num_object_copies":3934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19111","reported_seq":19248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:35.555828+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:35.555828+0000","last_peered":"2024-09-05T13:21:35.555828+0000","last_clean":"2024-09-05T13:21:35.555828+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:35.555828+0000","last_undegraded":"2024-09-05T13:21:35.555828+0000","last_fullsized":"2024-09-05T13:21:35.555828+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15721,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19086","reported_seq":19226,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.815693+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:38.815693+0000","last_peered":"2024-09-05T13:21:38.815693+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:38.815693+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8220672,"num_objects":2007,"num_object_clones":0,"num_object_copies":4014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16740,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2444}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18891","reported_seq":19035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.517743+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:35.517743+0000","last_peered":"2024-09-05T13:21:35.517743+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:35.517743+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8269824,"num_objects":2019,"num_object_clones":0,"num_object_copies":4038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15258,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18982","reported_seq":27889,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.821636+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:38.821636+0000","last_peered":"2024-09-05T13:21:38.821636+0000","last_clean":"2024-09-05T13:21:38.821636+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:38.821636+0000","last_undegraded":"2024-09-05T13:21:38.821636+0000","last_fullsized":"2024-09-05T13:21:38.821636+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7901184,"num_objects":1929,"num_object_clones":0,"num_object_copies":3858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16577,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106886+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106886+0000","last_peered":"2024-09-05T13:19:36.106886+0000","last_clean":"2024-09-05T13:19:36.106886+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106886+0000","last_undegraded":"2024-09-05T13:19:36.106886+0000","last_fullsized":"2024-09-05T13:19:36.106886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18673","reported_seq":18802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.826007+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:38.826007+0000","last_peered":"2024-09-05T13:21:38.826007+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:38.826007+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2478}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18872","reported_seq":19018,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.580184+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:35.580184+0000","last_peered":"2024-09-05T13:21:35.580184+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:35.580184+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8216576,"num_objects":2006,"num_object_clones":0,"num_object_copies":4012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15023,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107104+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107104+0000","last_peered":"2024-09-05T13:19:36.107104+0000","last_clean":"2024-09-05T13:19:36.107104+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107104+0000","last_undegraded":"2024-09-05T13:19:36.107104+0000","last_fullsized":"2024-09-05T13:19:36.107104+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19059","reported_seq":19252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.590401+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:35.590401+0000","last_peered":"2024-09-05T13:21:35.590401+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:35.590401+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376320,"num_objects":2045,"num_object_clones":0,"num_object_copies":4090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14958,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19088","reported_seq":19232,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.552250+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:35.552250+0000","last_peered":"2024-09-05T13:21:35.552250+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:35.552250+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8159232,"num_objects":1992,"num_object_clones":0,"num_object_copies":3984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15218,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107184+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107184+0000","last_peered":"2024-09-05T13:19:36.107184+0000","last_clean":"2024-09-05T13:19:36.107184+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107184+0000","last_undegraded":"2024-09-05T13:19:36.107184+0000","last_fullsized":"2024-09-05T13:19:36.107184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19209","reported_seq":19355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:35.567836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:35.567836+0000","last_peered":"2024-09-05T13:21:35.567836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:35.567836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8122851,"num_objects":1984,"num_object_clones":0,"num_object_copies":3968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15877,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2581}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":130380283,"num_objects":31833,"num_object_clones":0,"num_object_copies":63666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259354,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10983,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":482365440,"data_stored":482285036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160936,"ondisk_log_size":160936,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540636,"kb_used_data":206284,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93831204,"statfs":{"total":96636764160,"available":96083152896,"internally_reserved":0,"allocated":211234816,"data_stored":210964814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,1,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624432,"kb_used_data":169376,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93747408,"statfs":{"total":96636764160,"available":95997345792,"internally_reserved":0,"allocated":173441024,"data_stored":173171017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546312,"kb_used_data":99512,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93825528,"statfs":{"total":96636764160,"available":96077340672,"internally_reserved":0,"allocated":101900288,"data_stored":101564788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171565056,"data_stored":171561443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100794368,"data_stored":100725262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":210006016,"data_stored":209998331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:41.228 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:41.795 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:41.796 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:41.821 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-05T13:21:41.724877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":124359182,"num_objects":30365,"num_object_clones":0,"num_object_copies":60730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260826,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10984,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1709896,"kb_used_data":459796,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281405624,"statfs":{"total":289910292480,"available":288159358976,"internally_reserved":0,"allocated":470831104,"data_stored":469955595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1280035846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,6,6,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25165824,"num_objects":-6144,"num_object_clones":0,"num_object_copies":-12288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004081"},"pg_stats":[{"pgid":"2.f","version":"47'18999","reported_seq":19080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.590548+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:40.590548+0000","last_peered":"2024-09-05T13:21:40.590548+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:40.590548+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8081408,"num_objects":1973,"num_object_clones":0,"num_object_copies":3946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18999,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2448}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18836","reported_seq":18946,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.820733+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:38.820733+0000","last_peered":"2024-09-05T13:21:38.820733+0000","last_clean":"2024-09-05T13:21:38.820733+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:38.820733+0000","last_undegraded":"2024-09-05T13:21:38.820733+0000","last_fullsized":"2024-09-05T13:21:38.820733+0000","mapping_epoch":35,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7856128,"num_objects":1918,"num_object_clones":0,"num_object_copies":3836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16846,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19228","reported_seq":23670,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.848069+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:38.848069+0000","last_peered":"2024-09-05T13:21:38.848069+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:38.848069+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056856,"num_objects":1968,"num_object_clones":0,"num_object_copies":3936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17039,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19055","reported_seq":19130,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.849314+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:38.849314+0000","last_peered":"2024-09-05T13:21:38.849314+0000","last_clean":"2024-09-05T13:21:38.849314+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:38.849314+0000","last_undegraded":"2024-09-05T13:21:38.849314+0000","last_fullsized":"2024-09-05T13:21:38.849314+0000","mapping_epoch":35,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8073216,"num_objects":1971,"num_object_clones":0,"num_object_copies":3942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19055,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19039","reported_seq":19179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:40.636105+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:40.636105+0000","last_peered":"2024-09-05T13:21:40.636105+0000","last_clean":"2024-09-05T13:21:40.636105+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:40.636105+0000","last_undegraded":"2024-09-05T13:21:40.636105+0000","last_fullsized":"2024-09-05T13:21:40.636105+0000","mapping_epoch":34,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7553024,"num_objects":1844,"num_object_clones":0,"num_object_copies":3688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15579,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19275","reported_seq":19419,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.624027+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:40.624027+0000","last_peered":"2024-09-05T13:21:40.624027+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:40.624027+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7700480,"num_objects":1880,"num_object_clones":0,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15730,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18790","reported_seq":27821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:39.922169+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:39.922169+0000","last_peered":"2024-09-05T13:21:39.922169+0000","last_clean":"2024-09-05T13:21:39.922169+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:39.922169+0000","last_undegraded":"2024-09-05T13:21:39.922169+0000","last_fullsized":"2024-09-05T13:21:39.922169+0000","mapping_epoch":46,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7528448,"num_objects":1838,"num_object_clones":0,"num_object_copies":3676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15109,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19250","reported_seq":19388,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:40.615894+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:40.615894+0000","last_peered":"2024-09-05T13:21:40.615894+0000","last_clean":"2024-09-05T13:21:40.615894+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:40.615894+0000","last_undegraded":"2024-09-05T13:21:40.615894+0000","last_fullsized":"2024-09-05T13:21:40.615894+0000","mapping_epoch":34,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7593984,"num_objects":1854,"num_object_clones":0,"num_object_copies":3708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15860,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19086","reported_seq":19226,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.815693+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:38.815693+0000","last_peered":"2024-09-05T13:21:38.815693+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:38.815693+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8220672,"num_objects":2007,"num_object_clones":0,"num_object_copies":4014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16740,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2444}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19048","reported_seq":19192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.595472+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:40.595472+0000","last_peered":"2024-09-05T13:21:40.595472+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:40.595472+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7626752,"num_objects":1862,"num_object_clones":0,"num_object_copies":3724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15415,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2592}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18982","reported_seq":27889,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.821636+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:38.821636+0000","last_peered":"2024-09-05T13:21:38.821636+0000","last_clean":"2024-09-05T13:21:38.821636+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:38.821636+0000","last_undegraded":"2024-09-05T13:21:38.821636+0000","last_fullsized":"2024-09-05T13:21:38.821636+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7901184,"num_objects":1929,"num_object_clones":0,"num_object_copies":3858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16577,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18673","reported_seq":18802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.826007+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:38.826007+0000","last_peered":"2024-09-05T13:21:38.826007+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:38.826007+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2478}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19033","reported_seq":19179,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.627481+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:40.627481+0000","last_peered":"2024-09-05T13:21:40.627481+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:40.627481+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7557120,"num_objects":1845,"num_object_clones":0,"num_object_copies":3690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15184,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19216","reported_seq":19405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.598845+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:40.598845+0000","last_peered":"2024-09-05T13:21:40.598845+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:40.598845+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7733248,"num_objects":1888,"num_object_clones":0,"num_object_copies":3776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15115,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19229","reported_seq":19374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.556226+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:40.556226+0000","last_peered":"2024-09-05T13:21:40.556226+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:40.556226+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15359,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19367","reported_seq":19513,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.554473+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:40.554473+0000","last_peered":"2024-09-05T13:21:40.554473+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:40.554473+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7475683,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16035,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2739}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":124359163,"num_objects":30363,"num_object_clones":0,"num_object_copies":60726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260824,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10983,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":467890176,"data_stored":467809772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540636,"kb_used_data":206284,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93831204,"statfs":{"total":96636764160,"available":96083152896,"internally_reserved":0,"allocated":211234816,"data_stored":210964814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,1,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625820,"kb_used_data":161032,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93746020,"statfs":{"total":96636764160,"available":95995924480,"internally_reserved":0,"allocated":164896768,"data_stored":164626761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543440,"kb_used_data":92480,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93828400,"statfs":{"total":96636764160,"available":96080281600,"internally_reserved":0,"allocated":94699520,"data_stored":94364020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164372480,"data_stored":164368867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":93511680,"data_stored":93442574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":210006016,"data_stored":209998331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:41.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:41.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18999", 'reported_seq': 19080, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.590548+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:40.590548+0000', 'last_peered': '2024-09-05T13:21:40.590548+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:40.590548+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8081408, 'num_objects': 1973, 'num_object_clones': 0, 'num_object_copies': 3946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18999, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2448}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:41.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:41.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'19228", 'reported_seq': 23670, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:38.848069+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:38.848069+0000', 'last_peered': '2024-09-05T13:21:38.848069+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:38.848069+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8056856, 'num_objects': 1968, 'num_object_clones': 0, 'num_object_copies': 3936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2340, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1968, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17039, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2153, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -372}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:41.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:41.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'19275", 'reported_seq': 19419, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.624027+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:40.624027+0000', 'last_peered': '2024-09-05T13:21:40.624027+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:40.624027+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7700480, 'num_objects': 1880, 'num_object_clones': 0, 'num_object_copies': 3760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1880, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15730, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2571}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:41.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:41.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'19086", 'reported_seq': 19226, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:38.815693+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:38.815693+0000', 'last_peered': '2024-09-05T13:21:38.815693+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:38.815693+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8220672, 'num_objects': 2007, 'num_object_clones': 0, 'num_object_copies': 4014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16740, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:41.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:41.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'19048", 'reported_seq': 19192, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.595472+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:40.595472+0000', 'last_peered': '2024-09-05T13:21:40.595472+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:40.595472+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7626752, 'num_objects': 1862, 'num_object_clones': 0, 'num_object_copies': 3724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15415, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2592}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:41.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:41.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'18673", 'reported_seq': 18802, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:38.826007+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:38.826007+0000', 'last_peered': '2024-09-05T13:21:38.826007+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:38.826007+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7819264, 'num_objects': 1909, 'num_object_clones': 0, 'num_object_copies': 3818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1909, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16182, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2478}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:41.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:41.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'19033", 'reported_seq': 19179, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.627481+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:40.627481+0000', 'last_peered': '2024-09-05T13:21:40.627481+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:40.627481+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7557120, 'num_objects': 1845, 'num_object_clones': 0, 'num_object_copies': 3690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15184, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:41.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:41.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'19216", 'reported_seq': 19405, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.598845+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:40.598845+0000', 'last_peered': '2024-09-05T13:21:40.598845+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:40.598845+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7733248, 'num_objects': 1888, 'num_object_clones': 0, 'num_object_copies': 3776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15115, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:41.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:41.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'19229", 'reported_seq': 19374, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.556226+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:40.556226+0000', 'last_peered': '2024-09-05T13:21:40.556226+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:40.556226+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7581696, 'num_objects': 1851, 'num_object_clones': 0, 'num_object_copies': 3702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15359, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:41.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:41.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'19367", 'reported_seq': 19513, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.554473+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:40.554473+0000', 'last_peered': '2024-09-05T13:21:40.554473+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:40.554473+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7475683, 'num_objects': 1826, 'num_object_clones': 0, 'num_object_copies': 3652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16035, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2739}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:41.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:42.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:42.513 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42515/60730 objects degraded (70.007%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":629,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30365,"data_bytes":124359182,"bytes_used":1750933504,"bytes_avail":288159358976,"bytes_total":289910292480,"degraded_objects":42515,"degraded_total":60730,"degraded_ratio":0.70006586530545034,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":511},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:42.549 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:42.549 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:43.082 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:43.113 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-05T13:21:41.724877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":124359182,"num_objects":30365,"num_object_clones":0,"num_object_copies":60730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260826,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10984,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1709896,"kb_used_data":459796,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281405624,"statfs":{"total":289910292480,"available":288159358976,"internally_reserved":0,"allocated":470831104,"data_stored":469955595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1280035846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,6,6,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25165824,"num_objects":-6144,"num_object_clones":0,"num_object_copies":-12288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6144,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004081"},"pg_stats":[{"pgid":"2.f","version":"47'18999","reported_seq":19080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.590548+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:40.590548+0000","last_peered":"2024-09-05T13:21:40.590548+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:40.590548+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8081408,"num_objects":1973,"num_object_clones":0,"num_object_copies":3946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18999,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2448}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18836","reported_seq":18946,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.820733+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:38.820733+0000","last_peered":"2024-09-05T13:21:38.820733+0000","last_clean":"2024-09-05T13:21:38.820733+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:38.820733+0000","last_undegraded":"2024-09-05T13:21:38.820733+0000","last_fullsized":"2024-09-05T13:21:38.820733+0000","mapping_epoch":35,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7856128,"num_objects":1918,"num_object_clones":0,"num_object_copies":3836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16846,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19228","reported_seq":23670,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.848069+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:38.848069+0000","last_peered":"2024-09-05T13:21:38.848069+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:38.848069+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8056856,"num_objects":1968,"num_object_clones":0,"num_object_copies":3936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17039,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19055","reported_seq":19130,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.849314+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:38.849314+0000","last_peered":"2024-09-05T13:21:38.849314+0000","last_clean":"2024-09-05T13:21:38.849314+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:38.849314+0000","last_undegraded":"2024-09-05T13:21:38.849314+0000","last_fullsized":"2024-09-05T13:21:38.849314+0000","mapping_epoch":35,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8073216,"num_objects":1971,"num_object_clones":0,"num_object_copies":3942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19055,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19039","reported_seq":19179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:40.636105+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:40.636105+0000","last_peered":"2024-09-05T13:21:40.636105+0000","last_clean":"2024-09-05T13:21:40.636105+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:40.636105+0000","last_undegraded":"2024-09-05T13:21:40.636105+0000","last_fullsized":"2024-09-05T13:21:40.636105+0000","mapping_epoch":34,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7553024,"num_objects":1844,"num_object_clones":0,"num_object_copies":3688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15579,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19275","reported_seq":19419,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.624027+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:40.624027+0000","last_peered":"2024-09-05T13:21:40.624027+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:40.624027+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7700480,"num_objects":1880,"num_object_clones":0,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15730,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18790","reported_seq":27821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:39.922169+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:39.922169+0000","last_peered":"2024-09-05T13:21:39.922169+0000","last_clean":"2024-09-05T13:21:39.922169+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:39.922169+0000","last_undegraded":"2024-09-05T13:21:39.922169+0000","last_fullsized":"2024-09-05T13:21:39.922169+0000","mapping_epoch":46,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7528448,"num_objects":1838,"num_object_clones":0,"num_object_copies":3676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15109,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19250","reported_seq":19388,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:40.615894+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:40.615894+0000","last_peered":"2024-09-05T13:21:40.615894+0000","last_clean":"2024-09-05T13:21:40.615894+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:40.615894+0000","last_undegraded":"2024-09-05T13:21:40.615894+0000","last_fullsized":"2024-09-05T13:21:40.615894+0000","mapping_epoch":34,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7593984,"num_objects":1854,"num_object_clones":0,"num_object_copies":3708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15860,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19086","reported_seq":19226,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.815693+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:38.815693+0000","last_peered":"2024-09-05T13:21:38.815693+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:38.815693+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8220672,"num_objects":2007,"num_object_clones":0,"num_object_copies":4014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16740,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2444}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19048","reported_seq":19192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.595472+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:40.595472+0000","last_peered":"2024-09-05T13:21:40.595472+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:40.595472+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7626752,"num_objects":1862,"num_object_clones":0,"num_object_copies":3724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15415,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2592}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18982","reported_seq":27889,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:38.821636+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:38.821636+0000","last_peered":"2024-09-05T13:21:38.821636+0000","last_clean":"2024-09-05T13:21:38.821636+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:38.821636+0000","last_undegraded":"2024-09-05T13:21:38.821636+0000","last_fullsized":"2024-09-05T13:21:38.821636+0000","mapping_epoch":44,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7901184,"num_objects":1929,"num_object_clones":0,"num_object_copies":3858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16577,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18673","reported_seq":18802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:38.826007+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:38.826007+0000","last_peered":"2024-09-05T13:21:38.826007+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:38.826007+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2478}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19033","reported_seq":19179,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.627481+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:40.627481+0000","last_peered":"2024-09-05T13:21:40.627481+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:40.627481+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7557120,"num_objects":1845,"num_object_clones":0,"num_object_copies":3690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15184,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19216","reported_seq":19405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.598845+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:40.598845+0000","last_peered":"2024-09-05T13:21:40.598845+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:40.598845+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7733248,"num_objects":1888,"num_object_clones":0,"num_object_copies":3776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15115,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19229","reported_seq":19374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.556226+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:40.556226+0000","last_peered":"2024-09-05T13:21:40.556226+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:40.556226+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15359,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19367","reported_seq":19513,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.554473+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:40.554473+0000","last_peered":"2024-09-05T13:21:40.554473+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:40.554473+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7475683,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16035,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2739}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":124359163,"num_objects":30363,"num_object_clones":0,"num_object_copies":60726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260824,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10983,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":467890176,"data_stored":467809772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540636,"kb_used_data":206284,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93831204,"statfs":{"total":96636764160,"available":96083152896,"internally_reserved":0,"allocated":211234816,"data_stored":210964814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,1,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625820,"kb_used_data":161032,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93746020,"statfs":{"total":96636764160,"available":95995924480,"internally_reserved":0,"allocated":164896768,"data_stored":164626761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543440,"kb_used_data":92480,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93828400,"statfs":{"total":96636764160,"available":96080281600,"internally_reserved":0,"allocated":94699520,"data_stored":94364020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164372480,"data_stored":164368867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":93511680,"data_stored":93442574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":210006016,"data_stored":209998331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:46.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:46.693 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:46.695 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:46.721 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-05T13:21:45.726050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":120427022,"num_objects":29405,"num_object_clones":0,"num_object_copies":58810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261786,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11084,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1694308,"kb_used_data":448368,"kb_used_omap":12,"kb_used_meta":1245875,"kb_avail":281421212,"statfs":{"total":289910292480,"available":288175321088,"internally_reserved":0,"allocated":459128832,"data_stored":458253323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1275776006},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,10,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23126016,"num_objects":-5646,"num_object_clones":0,"num_object_copies":-11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5646,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003861"},"pg_stats":[{"pgid":"2.f","version":"47'18999","reported_seq":19080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.590548+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:40.590548+0000","last_peered":"2024-09-05T13:21:40.590548+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:40.590548+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8081408,"num_objects":1973,"num_object_clones":0,"num_object_copies":3946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18999,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2448}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18974","reported_seq":19084,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.879755+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:43.879755+0000","last_peered":"2024-09-05T13:21:43.879755+0000","last_clean":"2024-09-05T13:21:43.879755+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:43.879755+0000","last_undegraded":"2024-09-05T13:21:43.879755+0000","last_fullsized":"2024-09-05T13:21:43.879755+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290880,"num_objects":1780,"num_object_clones":0,"num_object_copies":3560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16984,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19372","reported_seq":24010,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.878088+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:43.878088+0000","last_peered":"2024-09-05T13:21:43.878088+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:43.878088+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7467032,"num_objects":1824,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17183,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19190","reported_seq":19266,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.833068+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:43.833068+0000","last_peered":"2024-09-05T13:21:43.833068+0000","last_clean":"2024-09-05T13:21:43.833068+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:43.833068+0000","last_undegraded":"2024-09-05T13:21:43.833068+0000","last_fullsized":"2024-09-05T13:21:43.833068+0000","mapping_epoch":35,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7520256,"num_objects":1836,"num_object_clones":0,"num_object_copies":3672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19190,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19039","reported_seq":19179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:40.636105+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:40.636105+0000","last_peered":"2024-09-05T13:21:40.636105+0000","last_clean":"2024-09-05T13:21:40.636105+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:40.636105+0000","last_undegraded":"2024-09-05T13:21:40.636105+0000","last_fullsized":"2024-09-05T13:21:40.636105+0000","mapping_epoch":34,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7553024,"num_objects":1844,"num_object_clones":0,"num_object_copies":3688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15579,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19275","reported_seq":19419,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.624027+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:40.624027+0000","last_peered":"2024-09-05T13:21:40.624027+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:40.624027+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7700480,"num_objects":1880,"num_object_clones":0,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15730,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18920","reported_seq":27952,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:45.022652+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:45.022652+0000","last_peered":"2024-09-05T13:21:45.022652+0000","last_clean":"2024-09-05T13:21:45.022652+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:45.022652+0000","last_undegraded":"2024-09-05T13:21:45.022652+0000","last_fullsized":"2024-09-05T13:21:45.022652+0000","mapping_epoch":46,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15239,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19250","reported_seq":19388,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:40.615894+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:40.615894+0000","last_peered":"2024-09-05T13:21:40.615894+0000","last_clean":"2024-09-05T13:21:40.615894+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:40.615894+0000","last_undegraded":"2024-09-05T13:21:40.615894+0000","last_fullsized":"2024-09-05T13:21:40.615894+0000","mapping_epoch":34,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7593984,"num_objects":1854,"num_object_clones":0,"num_object_copies":3708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15860,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19212","reported_seq":19353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.819092+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:43.819092+0000","last_peered":"2024-09-05T13:21:43.819092+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:43.819092+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7704576,"num_objects":1881,"num_object_clones":0,"num_object_copies":3762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16866,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2570}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19048","reported_seq":19192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.595472+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:40.595472+0000","last_peered":"2024-09-05T13:21:40.595472+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:40.595472+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7626752,"num_objects":1862,"num_object_clones":0,"num_object_copies":3724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15415,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2592}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19139","reported_seq":28046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.866559+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:43.866559+0000","last_peered":"2024-09-05T13:21:43.866559+0000","last_clean":"2024-09-05T13:21:43.866559+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:43.866559+0000","last_undegraded":"2024-09-05T13:21:43.866559+0000","last_fullsized":"2024-09-05T13:21:43.866559+0000","mapping_epoch":44,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7258112,"num_objects":1772,"num_object_clones":0,"num_object_copies":3544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16734,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18803","reported_seq":18932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.842422+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:43.842422+0000","last_peered":"2024-09-05T13:21:43.842422+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:43.842422+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7286784,"num_objects":1779,"num_object_clones":0,"num_object_copies":3558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16312,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2608}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19033","reported_seq":19179,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.627481+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:40.627481+0000","last_peered":"2024-09-05T13:21:40.627481+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:40.627481+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7557120,"num_objects":1845,"num_object_clones":0,"num_object_copies":3690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15184,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19216","reported_seq":19405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.598845+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:40.598845+0000","last_peered":"2024-09-05T13:21:40.598845+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:40.598845+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7733248,"num_objects":1888,"num_object_clones":0,"num_object_copies":3776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15115,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19229","reported_seq":19374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.556226+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:40.556226+0000","last_peered":"2024-09-05T13:21:40.556226+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:40.556226+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15359,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19367","reported_seq":19513,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.554473+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:40.554473+0000","last_peered":"2024-09-05T13:21:40.554473+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:40.554473+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7475683,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16035,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2739}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":120427003,"num_objects":29403,"num_object_clones":0,"num_object_copies":58806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261784,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11083,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":455942144,"data_stored":455861740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540228,"kb_used_data":201716,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93831612,"statfs":{"total":96636764160,"available":96083570688,"internally_reserved":0,"allocated":206557184,"data_stored":206287182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625820,"kb_used_data":161032,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93746020,"statfs":{"total":96636764160,"available":95995924480,"internally_reserved":0,"allocated":164896768,"data_stored":164626761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528260,"kb_used_data":85620,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93843580,"statfs":{"total":96636764160,"available":96095825920,"internally_reserved":0,"allocated":87674880,"data_stored":87339380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164372480,"data_stored":164368867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86183936,"data_stored":86114830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205385728,"data_stored":205378043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:46.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:47.271 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:47.296 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-05T13:21:45.726050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":120427022,"num_objects":29405,"num_object_clones":0,"num_object_copies":58810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261786,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11084,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1694308,"kb_used_data":448368,"kb_used_omap":12,"kb_used_meta":1245875,"kb_avail":281421212,"statfs":{"total":289910292480,"available":288175321088,"internally_reserved":0,"allocated":459128832,"data_stored":458253323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1275776006},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,10,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23126016,"num_objects":-5646,"num_object_clones":0,"num_object_copies":-11292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5646,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003861"},"pg_stats":[{"pgid":"2.f","version":"47'18999","reported_seq":19080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.590548+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:40.590548+0000","last_peered":"2024-09-05T13:21:40.590548+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:40.590548+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8081408,"num_objects":1973,"num_object_clones":0,"num_object_copies":3946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18999,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2448}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18974","reported_seq":19084,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.879755+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:43.879755+0000","last_peered":"2024-09-05T13:21:43.879755+0000","last_clean":"2024-09-05T13:21:43.879755+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:43.879755+0000","last_undegraded":"2024-09-05T13:21:43.879755+0000","last_fullsized":"2024-09-05T13:21:43.879755+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290880,"num_objects":1780,"num_object_clones":0,"num_object_copies":3560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16984,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19372","reported_seq":24010,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.878088+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:43.878088+0000","last_peered":"2024-09-05T13:21:43.878088+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:43.878088+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7467032,"num_objects":1824,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17183,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19190","reported_seq":19266,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.833068+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:43.833068+0000","last_peered":"2024-09-05T13:21:43.833068+0000","last_clean":"2024-09-05T13:21:43.833068+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:43.833068+0000","last_undegraded":"2024-09-05T13:21:43.833068+0000","last_fullsized":"2024-09-05T13:21:43.833068+0000","mapping_epoch":35,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7520256,"num_objects":1836,"num_object_clones":0,"num_object_copies":3672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19190,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19039","reported_seq":19179,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:40.636105+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:40.636105+0000","last_peered":"2024-09-05T13:21:40.636105+0000","last_clean":"2024-09-05T13:21:40.636105+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:40.636105+0000","last_undegraded":"2024-09-05T13:21:40.636105+0000","last_fullsized":"2024-09-05T13:21:40.636105+0000","mapping_epoch":34,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7553024,"num_objects":1844,"num_object_clones":0,"num_object_copies":3688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15579,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19275","reported_seq":19419,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.624027+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:40.624027+0000","last_peered":"2024-09-05T13:21:40.624027+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:40.624027+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7700480,"num_objects":1880,"num_object_clones":0,"num_object_copies":3760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15730,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18920","reported_seq":27952,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:45.022652+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:45.022652+0000","last_peered":"2024-09-05T13:21:45.022652+0000","last_clean":"2024-09-05T13:21:45.022652+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:45.022652+0000","last_undegraded":"2024-09-05T13:21:45.022652+0000","last_fullsized":"2024-09-05T13:21:45.022652+0000","mapping_epoch":46,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15239,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19250","reported_seq":19388,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:40.615894+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:40.615894+0000","last_peered":"2024-09-05T13:21:40.615894+0000","last_clean":"2024-09-05T13:21:40.615894+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:40.615894+0000","last_undegraded":"2024-09-05T13:21:40.615894+0000","last_fullsized":"2024-09-05T13:21:40.615894+0000","mapping_epoch":34,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7593984,"num_objects":1854,"num_object_clones":0,"num_object_copies":3708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15860,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19212","reported_seq":19353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.819092+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:43.819092+0000","last_peered":"2024-09-05T13:21:43.819092+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:43.819092+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7704576,"num_objects":1881,"num_object_clones":0,"num_object_copies":3762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16866,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2570}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19048","reported_seq":19192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.595472+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:40.595472+0000","last_peered":"2024-09-05T13:21:40.595472+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:40.595472+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7626752,"num_objects":1862,"num_object_clones":0,"num_object_copies":3724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15415,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2592}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19139","reported_seq":28046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.866559+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:43.866559+0000","last_peered":"2024-09-05T13:21:43.866559+0000","last_clean":"2024-09-05T13:21:43.866559+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:43.866559+0000","last_undegraded":"2024-09-05T13:21:43.866559+0000","last_fullsized":"2024-09-05T13:21:43.866559+0000","mapping_epoch":44,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7258112,"num_objects":1772,"num_object_clones":0,"num_object_copies":3544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16734,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18803","reported_seq":18932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.842422+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:43.842422+0000","last_peered":"2024-09-05T13:21:43.842422+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:43.842422+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7286784,"num_objects":1779,"num_object_clones":0,"num_object_copies":3558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16312,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2608}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19033","reported_seq":19179,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.627481+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:40.627481+0000","last_peered":"2024-09-05T13:21:40.627481+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:40.627481+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7557120,"num_objects":1845,"num_object_clones":0,"num_object_copies":3690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15184,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19216","reported_seq":19405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.598845+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:40.598845+0000","last_peered":"2024-09-05T13:21:40.598845+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:40.598845+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7733248,"num_objects":1888,"num_object_clones":0,"num_object_copies":3776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15115,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19229","reported_seq":19374,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.556226+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:40.556226+0000","last_peered":"2024-09-05T13:21:40.556226+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:40.556226+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15359,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19367","reported_seq":19513,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:40.554473+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:40.554473+0000","last_peered":"2024-09-05T13:21:40.554473+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:40.554473+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7475683,"num_objects":1826,"num_object_clones":0,"num_object_copies":3652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16035,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2739}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":120427003,"num_objects":29403,"num_object_clones":0,"num_object_copies":58806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":261784,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11083,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":455942144,"data_stored":455861740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540228,"kb_used_data":201716,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93831612,"statfs":{"total":96636764160,"available":96083570688,"internally_reserved":0,"allocated":206557184,"data_stored":206287182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625820,"kb_used_data":161032,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93746020,"statfs":{"total":96636764160,"available":95995924480,"internally_reserved":0,"allocated":164896768,"data_stored":164626761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528260,"kb_used_data":85620,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93843580,"statfs":{"total":96636764160,"available":96095825920,"internally_reserved":0,"allocated":87674880,"data_stored":87339380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164372480,"data_stored":164368867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86183936,"data_stored":86114830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205385728,"data_stored":205378043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:47.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:47.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'18999", 'reported_seq': 19080, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.590548+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:40.590548+0000', 'last_peered': '2024-09-05T13:21:40.590548+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:40.590548+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8081408, 'num_objects': 1973, 'num_object_clones': 0, 'num_object_copies': 3946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18999, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2448}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:47.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:47.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'19372", 'reported_seq': 24010, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:43.878088+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:43.878088+0000', 'last_peered': '2024-09-05T13:21:43.878088+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:43.878088+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7467032, 'num_objects': 1824, 'num_object_clones': 0, 'num_object_copies': 3648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2240, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1824, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17183, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2253, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -416}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:47.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:47.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'19275", 'reported_seq': 19419, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.624027+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:40.624027+0000', 'last_peered': '2024-09-05T13:21:40.624027+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:40.624027+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7700480, 'num_objects': 1880, 'num_object_clones': 0, 'num_object_copies': 3760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1880, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15730, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2571}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:47.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:47.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'19212", 'reported_seq': 19353, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:43.819092+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:43.819092+0000', 'last_peered': '2024-09-05T13:21:43.819092+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:43.819092+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7704576, 'num_objects': 1881, 'num_object_clones': 0, 'num_object_copies': 3762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16866, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2570}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'19048", 'reported_seq': 19192, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.595472+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:40.595472+0000', 'last_peered': '2024-09-05T13:21:40.595472+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:40.595472+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7626752, 'num_objects': 1862, 'num_object_clones': 0, 'num_object_copies': 3724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15415, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2592}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'18803", 'reported_seq': 18932, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:43.842422+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:43.842422+0000', 'last_peered': '2024-09-05T13:21:43.842422+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:43.842422+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7286784, 'num_objects': 1779, 'num_object_clones': 0, 'num_object_copies': 3558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1779, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16312, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2608}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'19033", 'reported_seq': 19179, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.627481+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:40.627481+0000', 'last_peered': '2024-09-05T13:21:40.627481+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:40.627481+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7557120, 'num_objects': 1845, 'num_object_clones': 0, 'num_object_copies': 3690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15184, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'19216", 'reported_seq': 19405, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.598845+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:40.598845+0000', 'last_peered': '2024-09-05T13:21:40.598845+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:40.598845+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7733248, 'num_objects': 1888, 'num_object_clones': 0, 'num_object_copies': 3776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15115, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:47.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:47.300 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'19229", 'reported_seq': 19374, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.556226+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:40.556226+0000', 'last_peered': '2024-09-05T13:21:40.556226+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:40.556226+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7581696, 'num_objects': 1851, 'num_object_clones': 0, 'num_object_copies': 3702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15359, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:47.300 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:47.300 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'19367", 'reported_seq': 19513, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:40.554473+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:40.554473+0000', 'last_peered': '2024-09-05T13:21:40.554473+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:40.554473+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7475683, 'num_objects': 1826, 'num_object_clones': 0, 'num_object_copies': 3652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16035, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2739}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:47.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:47.998 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:47.998 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42415/58810 objects degraded (72.122%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":634,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":29405,"data_bytes":120427022,"bytes_used":1734971392,"bytes_avail":288175321088,"bytes_total":289910292480,"degraded_objects":42415,"degraded_total":58810,"degraded_ratio":0.72122088080258462,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":470},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:48.027 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:48.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:48.597 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:48.618 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-05T13:21:47.726855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":114938382,"num_objects":28065,"num_object_clones":0,"num_object_copies":56130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263126,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11084,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1691684,"kb_used_data":441588,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281423836,"statfs":{"total":289910292480,"available":288178008064,"internally_reserved":0,"allocated":452186112,"data_stored":451310603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1280035846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,12,5,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19066880,"num_objects":-4655,"num_object_clones":0,"num_object_copies":-9310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004104"},"pg_stats":[{"pgid":"2.f","version":"47'19158","reported_seq":19239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:45.707166+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:45.707166+0000","last_peered":"2024-09-05T13:21:45.707166+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:45.707166+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7430144,"num_objects":1814,"num_object_clones":0,"num_object_copies":3628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19158,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18974","reported_seq":19084,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.879755+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:43.879755+0000","last_peered":"2024-09-05T13:21:43.879755+0000","last_clean":"2024-09-05T13:21:43.879755+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:43.879755+0000","last_undegraded":"2024-09-05T13:21:43.879755+0000","last_fullsized":"2024-09-05T13:21:43.879755+0000","mapping_epoch":35,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7290880,"num_objects":1780,"num_object_clones":0,"num_object_copies":3560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16984,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19372","reported_seq":24010,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.878088+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:43.878088+0000","last_peered":"2024-09-05T13:21:43.878088+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:43.878088+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7467032,"num_objects":1824,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1824,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17183,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-416}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19190","reported_seq":19266,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.833068+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:43.833068+0000","last_peered":"2024-09-05T13:21:43.833068+0000","last_clean":"2024-09-05T13:21:43.833068+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:43.833068+0000","last_undegraded":"2024-09-05T13:21:43.833068+0000","last_fullsized":"2024-09-05T13:21:43.833068+0000","mapping_epoch":35,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7520256,"num_objects":1836,"num_object_clones":0,"num_object_copies":3672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19190,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19173","reported_seq":19313,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:45.658536+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:45.658536+0000","last_peered":"2024-09-05T13:21:45.658536+0000","last_clean":"2024-09-05T13:21:45.658536+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:45.658536+0000","last_undegraded":"2024-09-05T13:21:45.658536+0000","last_fullsized":"2024-09-05T13:21:45.658536+0000","mapping_epoch":34,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7004160,"num_objects":1710,"num_object_clones":0,"num_object_copies":3420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19452","reported_seq":19595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:45.736122+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:45.736122+0000","last_peered":"2024-09-05T13:21:45.736122+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:45.736122+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6975488,"num_objects":1703,"num_object_clones":0,"num_object_copies":3406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15907,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18920","reported_seq":27952,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:45.022652+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:45.022652+0000","last_peered":"2024-09-05T13:21:45.022652+0000","last_clean":"2024-09-05T13:21:45.022652+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:45.022652+0000","last_undegraded":"2024-09-05T13:21:45.022652+0000","last_fullsized":"2024-09-05T13:21:45.022652+0000","mapping_epoch":46,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15239,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19391","reported_seq":19529,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:45.665152+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:45.665152+0000","last_peered":"2024-09-05T13:21:45.665152+0000","last_clean":"2024-09-05T13:21:45.665152+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:45.665152+0000","last_undegraded":"2024-09-05T13:21:45.665152+0000","last_fullsized":"2024-09-05T13:21:45.665152+0000","mapping_epoch":34,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7016448,"num_objects":1713,"num_object_clones":0,"num_object_copies":3426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16001,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19212","reported_seq":19353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.819092+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:43.819092+0000","last_peered":"2024-09-05T13:21:43.819092+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:43.819092+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7704576,"num_objects":1881,"num_object_clones":0,"num_object_copies":3762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16866,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2570}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19214","reported_seq":19358,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:45.665311+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:45.665311+0000","last_peered":"2024-09-05T13:21:45.665311+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:45.665311+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6946816,"num_objects":1696,"num_object_clones":0,"num_object_copies":3392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15581,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19139","reported_seq":28046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:43.866559+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:43.866559+0000","last_peered":"2024-09-05T13:21:43.866559+0000","last_clean":"2024-09-05T13:21:43.866559+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:43.866559+0000","last_undegraded":"2024-09-05T13:21:43.866559+0000","last_fullsized":"2024-09-05T13:21:43.866559+0000","mapping_epoch":44,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7258112,"num_objects":1772,"num_object_clones":0,"num_object_copies":3544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16734,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18803","reported_seq":18932,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:43.842422+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:43.842422+0000","last_peered":"2024-09-05T13:21:43.842422+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:43.842422+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7286784,"num_objects":1779,"num_object_clones":0,"num_object_copies":3558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16312,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2608}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19167","reported_seq":19313,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:45.591152+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:45.591152+0000","last_peered":"2024-09-05T13:21:45.591152+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:45.591152+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7008256,"num_objects":1711,"num_object_clones":0,"num_object_copies":3422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15318,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19351","reported_seq":19545,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:45.718253+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:45.718253+0000","last_peered":"2024-09-05T13:21:45.718253+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:45.718253+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7180288,"num_objects":1753,"num_object_clones":0,"num_object_copies":3506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15250,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2691}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19386","reported_seq":19526,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:45.655767+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:45.655767+0000","last_peered":"2024-09-05T13:21:45.655767+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:45.655767+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6938624,"num_objects":1694,"num_object_clones":0,"num_object_copies":3388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15516,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2872}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19504","reported_seq":19650,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:45.664161+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:45.664161+0000","last_peered":"2024-09-05T13:21:45.664161+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:45.664161+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6914531,"num_objects":1689,"num_object_clones":0,"num_object_copies":3378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16172,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2876}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":114938363,"num_objects":28063,"num_object_clones":0,"num_object_copies":56126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42415,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263124,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11083,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":448741376,"data_stored":448660972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540228,"kb_used_data":201716,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93831612,"statfs":{"total":96636764160,"available":96083570688,"internally_reserved":0,"allocated":206557184,"data_stored":206287182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623196,"kb_used_data":154252,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93748644,"statfs":{"total":96636764160,"available":95998611456,"internally_reserved":0,"allocated":157954048,"data_stored":157684041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":480177834},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528260,"kb_used_data":85620,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93843580,"statfs":{"total":96636764160,"available":96095825920,"internally_reserved":0,"allocated":87674880,"data_stored":87339380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157171712,"data_stored":157168099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86183936,"data_stored":86114830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205385728,"data_stored":205378043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:51.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:52.186 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:52.188 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:52.213 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-05T13:21:51.728394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105460238,"num_objects":25751,"num_object_clones":0,"num_object_copies":51502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265440,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11187,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160623,"ondisk_log_size":160623,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1677512,"kb_used_data":423248,"kb_used_omap":12,"kb_used_meta":1254195,"kb_avail":281438008,"statfs":{"total":289910292480,"available":288192520192,"internally_reserved":0,"allocated":433405952,"data_stored":432530443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1284295686},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,6,6,9,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24920064,"num_objects":-6084,"num_object_clones":0,"num_object_copies":-12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6084,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004170"},"pg_stats":[{"pgid":"2.f","version":"47'19305","reported_seq":19385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.687393+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:50.687393+0000","last_peered":"2024-09-05T13:21:50.687393+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:50.687393+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6828032,"num_objects":1667,"num_object_clones":0,"num_object_copies":3334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19305,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19116","reported_seq":19227,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.906433+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:48.906433+0000","last_peered":"2024-09-05T13:21:48.906433+0000","last_clean":"2024-09-05T13:21:48.906433+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:48.906433+0000","last_undegraded":"2024-09-05T13:21:48.906433+0000","last_fullsized":"2024-09-05T13:21:48.906433+0000","mapping_epoch":35,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6709248,"num_objects":1638,"num_object_clones":0,"num_object_copies":3276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17126,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19515","reported_seq":24363,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.884054+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:48.884054+0000","last_peered":"2024-09-05T13:21:48.884054+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:48.884054+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6881304,"num_objects":1681,"num_object_clones":0,"num_object_copies":3362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17326,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19340","reported_seq":19415,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.845425+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:48.845425+0000","last_peered":"2024-09-05T13:21:48.845425+0000","last_clean":"2024-09-05T13:21:48.845425+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:48.845425+0000","last_undegraded":"2024-09-05T13:21:48.845425+0000","last_fullsized":"2024-09-05T13:21:48.845425+0000","mapping_epoch":35,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6905856,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19340,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19333","reported_seq":19474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.782190+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:50.782190+0000","last_peered":"2024-09-05T13:21:50.782190+0000","last_clean":"2024-09-05T13:21:50.782190+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:50.782190+0000","last_undegraded":"2024-09-05T13:21:50.782190+0000","last_fullsized":"2024-09-05T13:21:50.782190+0000","mapping_epoch":34,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6348800,"num_objects":1550,"num_object_clones":0,"num_object_copies":3100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15873,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19583","reported_seq":19727,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.717912+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:50.717912+0000","last_peered":"2024-09-05T13:21:50.717912+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:50.717912+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16038,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19056","reported_seq":28089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.105888+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:50.105888+0000","last_peered":"2024-09-05T13:21:50.105888+0000","last_clean":"2024-09-05T13:21:50.105888+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:50.105888+0000","last_undegraded":"2024-09-05T13:21:50.105888+0000","last_fullsized":"2024-09-05T13:21:50.105888+0000","mapping_epoch":46,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15375,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19552","reported_seq":19690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.745159+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:50.745159+0000","last_peered":"2024-09-05T13:21:50.745159+0000","last_clean":"2024-09-05T13:21:50.745159+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:50.745159+0000","last_undegraded":"2024-09-05T13:21:50.745159+0000","last_fullsized":"2024-09-05T13:21:50.745159+0000","mapping_epoch":34,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":1552,"num_object_clones":0,"num_object_copies":3104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16162,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19354","reported_seq":19495,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.897153+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:48.897153+0000","last_peered":"2024-09-05T13:21:48.897153+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:48.897153+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7122944,"num_objects":1739,"num_object_clones":0,"num_object_copies":3478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17008,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2712}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19346","reported_seq":19490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.703328+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:50.703328+0000","last_peered":"2024-09-05T13:21:50.703328+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:50.703328+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6406144,"num_objects":1564,"num_object_clones":0,"num_object_copies":3128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2890}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19284","reported_seq":28191,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.783961+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:48.783961+0000","last_peered":"2024-09-05T13:21:48.783961+0000","last_clean":"2024-09-05T13:21:48.783961+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:48.783961+0000","last_undegraded":"2024-09-05T13:21:48.783961+0000","last_fullsized":"2024-09-05T13:21:48.783961+0000","mapping_epoch":44,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6664192,"num_objects":1627,"num_object_clones":0,"num_object_copies":3254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18939","reported_seq":19068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.821135+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:48.821135+0000","last_peered":"2024-09-05T13:21:48.821135+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:48.821135+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16448,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2744}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19319","reported_seq":19464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.759438+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:50.759438+0000","last_peered":"2024-09-05T13:21:50.759438+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:50.759438+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15470,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19506","reported_seq":19700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.745092+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:50.745092+0000","last_peered":"2024-09-05T13:21:50.745092+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:50.745092+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15405,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2846}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19518","reported_seq":19663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.780062+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:50.780062+0000","last_peered":"2024-09-05T13:21:50.780062+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:50.780062+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6397952,"num_objects":1562,"num_object_clones":0,"num_object_copies":3124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15648,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3004}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19654","reported_seq":19800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.746992+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:50.746992+0000","last_peered":"2024-09-05T13:21:50.746992+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:50.746992+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6300131,"num_objects":1539,"num_object_clones":0,"num_object_copies":3078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16322,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3026}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":105460219,"num_objects":25749,"num_object_clones":0,"num_object_copies":51498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265438,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11186,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":429821952,"data_stored":429741548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160620,"ondisk_log_size":160620,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539728,"kb_used_data":197056,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93832112,"statfs":{"total":96636764160,"available":96084082688,"internally_reserved":0,"allocated":201785344,"data_stored":201515342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608072,"kb_used_data":147440,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93763768,"statfs":{"total":96636764160,"available":96014098432,"internally_reserved":0,"allocated":150978560,"data_stored":150708553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,0,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529712,"kb_used_data":78752,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93842128,"statfs":{"total":96636764160,"available":96094339072,"internally_reserved":0,"allocated":80642048,"data_stored":80306548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150024192,"data_stored":150020579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":79138816,"data_stored":79069710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":200658944,"data_stored":200651259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:52.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:52.757 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:52.757 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:52.781 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-05T13:21:51.728394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105460238,"num_objects":25751,"num_object_clones":0,"num_object_copies":51502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265440,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11187,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160623,"ondisk_log_size":160623,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1677512,"kb_used_data":423248,"kb_used_omap":12,"kb_used_meta":1254195,"kb_avail":281438008,"statfs":{"total":289910292480,"available":288192520192,"internally_reserved":0,"allocated":433405952,"data_stored":432530443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1284295686},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,6,6,9,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24920064,"num_objects":-6084,"num_object_clones":0,"num_object_copies":-12168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6084,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004170"},"pg_stats":[{"pgid":"2.f","version":"47'19305","reported_seq":19385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.687393+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:50.687393+0000","last_peered":"2024-09-05T13:21:50.687393+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:50.687393+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6828032,"num_objects":1667,"num_object_clones":0,"num_object_copies":3334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19305,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19116","reported_seq":19227,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.906433+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:48.906433+0000","last_peered":"2024-09-05T13:21:48.906433+0000","last_clean":"2024-09-05T13:21:48.906433+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:48.906433+0000","last_undegraded":"2024-09-05T13:21:48.906433+0000","last_fullsized":"2024-09-05T13:21:48.906433+0000","mapping_epoch":35,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6709248,"num_objects":1638,"num_object_clones":0,"num_object_copies":3276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17126,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19515","reported_seq":24363,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.884054+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:48.884054+0000","last_peered":"2024-09-05T13:21:48.884054+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:48.884054+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6881304,"num_objects":1681,"num_object_clones":0,"num_object_copies":3362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17326,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19340","reported_seq":19415,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.845425+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:48.845425+0000","last_peered":"2024-09-05T13:21:48.845425+0000","last_clean":"2024-09-05T13:21:48.845425+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:48.845425+0000","last_undegraded":"2024-09-05T13:21:48.845425+0000","last_fullsized":"2024-09-05T13:21:48.845425+0000","mapping_epoch":35,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6905856,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19340,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19333","reported_seq":19474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.782190+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:50.782190+0000","last_peered":"2024-09-05T13:21:50.782190+0000","last_clean":"2024-09-05T13:21:50.782190+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:50.782190+0000","last_undegraded":"2024-09-05T13:21:50.782190+0000","last_fullsized":"2024-09-05T13:21:50.782190+0000","mapping_epoch":34,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6348800,"num_objects":1550,"num_object_clones":0,"num_object_copies":3100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15873,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19583","reported_seq":19727,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.717912+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:50.717912+0000","last_peered":"2024-09-05T13:21:50.717912+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:50.717912+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16038,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19056","reported_seq":28089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.105888+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:50.105888+0000","last_peered":"2024-09-05T13:21:50.105888+0000","last_clean":"2024-09-05T13:21:50.105888+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:50.105888+0000","last_undegraded":"2024-09-05T13:21:50.105888+0000","last_fullsized":"2024-09-05T13:21:50.105888+0000","mapping_epoch":46,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15375,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19552","reported_seq":19690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.745159+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:50.745159+0000","last_peered":"2024-09-05T13:21:50.745159+0000","last_clean":"2024-09-05T13:21:50.745159+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:50.745159+0000","last_undegraded":"2024-09-05T13:21:50.745159+0000","last_fullsized":"2024-09-05T13:21:50.745159+0000","mapping_epoch":34,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":1552,"num_object_clones":0,"num_object_copies":3104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16162,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19354","reported_seq":19495,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.897153+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:48.897153+0000","last_peered":"2024-09-05T13:21:48.897153+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:48.897153+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7122944,"num_objects":1739,"num_object_clones":0,"num_object_copies":3478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17008,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2712}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19346","reported_seq":19490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.703328+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:50.703328+0000","last_peered":"2024-09-05T13:21:50.703328+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:50.703328+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6406144,"num_objects":1564,"num_object_clones":0,"num_object_copies":3128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2890}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19284","reported_seq":28191,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.783961+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:48.783961+0000","last_peered":"2024-09-05T13:21:48.783961+0000","last_clean":"2024-09-05T13:21:48.783961+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:48.783961+0000","last_undegraded":"2024-09-05T13:21:48.783961+0000","last_fullsized":"2024-09-05T13:21:48.783961+0000","mapping_epoch":44,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6664192,"num_objects":1627,"num_object_clones":0,"num_object_copies":3254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18939","reported_seq":19068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.821135+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:48.821135+0000","last_peered":"2024-09-05T13:21:48.821135+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:48.821135+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16448,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2744}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19319","reported_seq":19464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.759438+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:50.759438+0000","last_peered":"2024-09-05T13:21:50.759438+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:50.759438+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15470,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19506","reported_seq":19700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.745092+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:50.745092+0000","last_peered":"2024-09-05T13:21:50.745092+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:50.745092+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15405,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2846}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19518","reported_seq":19663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.780062+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:50.780062+0000","last_peered":"2024-09-05T13:21:50.780062+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:50.780062+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6397952,"num_objects":1562,"num_object_clones":0,"num_object_copies":3124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15648,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3004}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19654","reported_seq":19800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.746992+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:50.746992+0000","last_peered":"2024-09-05T13:21:50.746992+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:50.746992+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6300131,"num_objects":1539,"num_object_clones":0,"num_object_copies":3078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16322,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3026}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":105460219,"num_objects":25749,"num_object_clones":0,"num_object_copies":51498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265438,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11186,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":429821952,"data_stored":429741548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160620,"ondisk_log_size":160620,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539728,"kb_used_data":197056,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93832112,"statfs":{"total":96636764160,"available":96084082688,"internally_reserved":0,"allocated":201785344,"data_stored":201515342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608072,"kb_used_data":147440,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93763768,"statfs":{"total":96636764160,"available":96014098432,"internally_reserved":0,"allocated":150978560,"data_stored":150708553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,0,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529712,"kb_used_data":78752,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93842128,"statfs":{"total":96636764160,"available":96094339072,"internally_reserved":0,"allocated":80642048,"data_stored":80306548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150024192,"data_stored":150020579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":79138816,"data_stored":79069710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":200658944,"data_stored":200651259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:52.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:52.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'19305", 'reported_seq': 19385, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:50.687393+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:50.687393+0000', 'last_peered': '2024-09-05T13:21:50.687393+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:50.687393+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6828032, 'num_objects': 1667, 'num_object_clones': 0, 'num_object_copies': 3334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19305, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2754}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:52.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:52.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'19515", 'reported_seq': 24363, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:48.884054+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:48.884054+0000', 'last_peered': '2024-09-05T13:21:48.884054+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:48.884054+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6881304, 'num_objects': 1681, 'num_object_clones': 0, 'num_object_copies': 3362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2137, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1681, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17326, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2356, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:52.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:52.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'19583", 'reported_seq': 19727, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:50.717912+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:50.717912+0000', 'last_peered': '2024-09-05T13:21:50.717912+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:50.717912+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6438912, 'num_objects': 1572, 'num_object_clones': 0, 'num_object_copies': 3144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16038, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:52.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:52.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'19354", 'reported_seq': 19495, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:48.897153+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:48.897153+0000', 'last_peered': '2024-09-05T13:21:48.897153+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:48.897153+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7122944, 'num_objects': 1739, 'num_object_clones': 0, 'num_object_copies': 3478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17008, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2712}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'19346", 'reported_seq': 19490, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:50.703328+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:50.703328+0000', 'last_peered': '2024-09-05T13:21:50.703328+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:50.703328+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6406144, 'num_objects': 1564, 'num_object_clones': 0, 'num_object_copies': 3128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1564, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15713, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2890}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'18939", 'reported_seq': 19068, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:48.821135+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:48.821135+0000', 'last_peered': '2024-09-05T13:21:48.821135+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:48.821135+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'8900", 'ondisk_log_start': "43'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6729728, 'num_objects': 1643, 'num_object_clones': 0, 'num_object_copies': 3286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16448, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2744}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'19319", 'reported_seq': 19464, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:50.759438+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:50.759438+0000', 'last_peered': '2024-09-05T13:21:50.759438+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:50.759438+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6385664, 'num_objects': 1559, 'num_object_clones': 0, 'num_object_copies': 3118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1559, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15470, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'19506", 'reported_seq': 19700, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:50.745092+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:50.745092+0000', 'last_peered': '2024-09-05T13:21:50.745092+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:50.745092+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6545408, 'num_objects': 1598, 'num_object_clones': 0, 'num_object_copies': 3196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15405, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2846}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:52.784 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'19518", 'reported_seq': 19663, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:50.780062+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:50.780062+0000', 'last_peered': '2024-09-05T13:21:50.780062+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:50.780062+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6397952, 'num_objects': 1562, 'num_object_clones': 0, 'num_object_copies': 3124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1562, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15648, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3004}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:52.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:52.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'19654", 'reported_seq': 19800, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:50.746992+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:50.746992+0000', 'last_peered': '2024-09-05T13:21:50.746992+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:50.746992+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6300131, 'num_objects': 1539, 'num_object_clones': 0, 'num_object_copies': 3078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1539, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16322, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3026}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:52.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:53.482 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:53.482 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42312/51502 objects degraded (82.156%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":640,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":25751,"data_bytes":105460238,"bytes_used":1717772288,"bytes_avail":288192520192,"bytes_total":289910292480,"degraded_objects":42312,"degraded_total":51502,"degraded_ratio":0.82156032775426202,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":506},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:53.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:53.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:54.081 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:54.084 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:54.107 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-05T13:21:53.729141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":105460238,"num_objects":25751,"num_object_clones":0,"num_object_copies":51502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265440,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11187,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160623,"ondisk_log_size":160623,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1677512,"kb_used_data":423248,"kb_used_omap":12,"kb_used_meta":1254195,"kb_avail":281438008,"statfs":{"total":289910292480,"available":288192520192,"internally_reserved":0,"allocated":433405952,"data_stored":432530443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1284295686},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,6,6,9,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18898944,"num_objects":-4614,"num_object_clones":0,"num_object_copies":-9228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4614,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004264"},"pg_stats":[{"pgid":"2.f","version":"47'19305","reported_seq":19385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.687393+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:50.687393+0000","last_peered":"2024-09-05T13:21:50.687393+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:50.687393+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6828032,"num_objects":1667,"num_object_clones":0,"num_object_copies":3334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19305,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19116","reported_seq":19227,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.906433+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:48.906433+0000","last_peered":"2024-09-05T13:21:48.906433+0000","last_clean":"2024-09-05T13:21:48.906433+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:48.906433+0000","last_undegraded":"2024-09-05T13:21:48.906433+0000","last_fullsized":"2024-09-05T13:21:48.906433+0000","mapping_epoch":35,"log_start":"43'9100","ondisk_log_start":"43'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6709248,"num_objects":1638,"num_object_clones":0,"num_object_copies":3276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17126,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19515","reported_seq":24363,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.884054+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:48.884054+0000","last_peered":"2024-09-05T13:21:48.884054+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:48.884054+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6881304,"num_objects":1681,"num_object_clones":0,"num_object_copies":3362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17326,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19340","reported_seq":19415,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.845425+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:48.845425+0000","last_peered":"2024-09-05T13:21:48.845425+0000","last_clean":"2024-09-05T13:21:48.845425+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:48.845425+0000","last_undegraded":"2024-09-05T13:21:48.845425+0000","last_fullsized":"2024-09-05T13:21:48.845425+0000","mapping_epoch":35,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6905856,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19340,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19333","reported_seq":19474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.782190+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:50.782190+0000","last_peered":"2024-09-05T13:21:50.782190+0000","last_clean":"2024-09-05T13:21:50.782190+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:50.782190+0000","last_undegraded":"2024-09-05T13:21:50.782190+0000","last_fullsized":"2024-09-05T13:21:50.782190+0000","mapping_epoch":34,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6348800,"num_objects":1550,"num_object_clones":0,"num_object_copies":3100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15873,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19583","reported_seq":19727,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.717912+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:50.717912+0000","last_peered":"2024-09-05T13:21:50.717912+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:50.717912+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16038,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19056","reported_seq":28089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.105888+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:50.105888+0000","last_peered":"2024-09-05T13:21:50.105888+0000","last_clean":"2024-09-05T13:21:50.105888+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:50.105888+0000","last_undegraded":"2024-09-05T13:21:50.105888+0000","last_fullsized":"2024-09-05T13:21:50.105888+0000","mapping_epoch":46,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15375,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19552","reported_seq":19690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.745159+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:50.745159+0000","last_peered":"2024-09-05T13:21:50.745159+0000","last_clean":"2024-09-05T13:21:50.745159+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:50.745159+0000","last_undegraded":"2024-09-05T13:21:50.745159+0000","last_fullsized":"2024-09-05T13:21:50.745159+0000","mapping_epoch":34,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":1552,"num_object_clones":0,"num_object_copies":3104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16162,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19354","reported_seq":19495,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.897153+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:48.897153+0000","last_peered":"2024-09-05T13:21:48.897153+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:48.897153+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7122944,"num_objects":1739,"num_object_clones":0,"num_object_copies":3478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17008,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2712}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19346","reported_seq":19490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.703328+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:50.703328+0000","last_peered":"2024-09-05T13:21:50.703328+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:50.703328+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6406144,"num_objects":1564,"num_object_clones":0,"num_object_copies":3128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2890}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19284","reported_seq":28191,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:48.783961+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:48.783961+0000","last_peered":"2024-09-05T13:21:48.783961+0000","last_clean":"2024-09-05T13:21:48.783961+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:48.783961+0000","last_undegraded":"2024-09-05T13:21:48.783961+0000","last_fullsized":"2024-09-05T13:21:48.783961+0000","mapping_epoch":44,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6664192,"num_objects":1627,"num_object_clones":0,"num_object_copies":3254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16879,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18939","reported_seq":19068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:48.821135+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:48.821135+0000","last_peered":"2024-09-05T13:21:48.821135+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:48.821135+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'8900","ondisk_log_start":"43'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6729728,"num_objects":1643,"num_object_clones":0,"num_object_copies":3286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16448,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2744}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19319","reported_seq":19464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.759438+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:50.759438+0000","last_peered":"2024-09-05T13:21:50.759438+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:50.759438+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15470,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19506","reported_seq":19700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.745092+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:50.745092+0000","last_peered":"2024-09-05T13:21:50.745092+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:50.745092+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15405,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2846}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19518","reported_seq":19663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.780062+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:50.780062+0000","last_peered":"2024-09-05T13:21:50.780062+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:50.780062+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6397952,"num_objects":1562,"num_object_clones":0,"num_object_copies":3124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15648,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3004}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19654","reported_seq":19800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.746992+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:50.746992+0000","last_peered":"2024-09-05T13:21:50.746992+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:50.746992+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6300131,"num_objects":1539,"num_object_clones":0,"num_object_copies":3078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16322,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3026}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":105460219,"num_objects":25749,"num_object_clones":0,"num_object_copies":51498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265438,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11186,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":429821952,"data_stored":429741548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160620,"ondisk_log_size":160620,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539728,"kb_used_data":197056,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93832112,"statfs":{"total":96636764160,"available":96084082688,"internally_reserved":0,"allocated":201785344,"data_stored":201515342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608072,"kb_used_data":147440,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93763768,"statfs":{"total":96636764160,"available":96014098432,"internally_reserved":0,"allocated":150978560,"data_stored":150708553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,0,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529712,"kb_used_data":78752,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93842128,"statfs":{"total":96636764160,"available":96094339072,"internally_reserved":0,"allocated":80642048,"data_stored":80306548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150024192,"data_stored":150020579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":79138816,"data_stored":79069710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":200658944,"data_stored":200651259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:57.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:57.678 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:57.681 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:57.700 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-05T13:21:55.729742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":101085710,"num_objects":24683,"num_object_clones":0,"num_object_copies":49366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24683,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266508,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11289,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160691,"ondisk_log_size":160691,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1674128,"kb_used_data":411544,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281441392,"statfs":{"total":289910292480,"available":288195985408,"internally_reserved":0,"allocated":421421056,"data_stored":420545547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1292815366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,5,5,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23273472,"num_objects":-5682,"num_object_clones":0,"num_object_copies":-11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5682,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004344"},"pg_stats":[{"pgid":"2.f","version":"47'19305","reported_seq":19385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.687393+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:50.687393+0000","last_peered":"2024-09-05T13:21:50.687393+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:50.687393+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6828032,"num_objects":1667,"num_object_clones":0,"num_object_copies":3334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19305,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19263","reported_seq":19374,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.908055+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:53.908055+0000","last_peered":"2024-09-05T13:21:53.908055+0000","last_clean":"2024-09-05T13:21:53.908055+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:53.908055+0000","last_undegraded":"2024-09-05T13:21:53.908055+0000","last_fullsized":"2024-09-05T13:21:53.908055+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17273,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19666","reported_seq":24717,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.875978+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:53.875978+0000","last_peered":"2024-09-05T13:21:53.875978+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:53.875978+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262808,"num_objects":1530,"num_object_clones":0,"num_object_copies":3060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1530,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17477,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19483","reported_seq":19559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.903975+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:53.903975+0000","last_peered":"2024-09-05T13:21:53.903975+0000","last_clean":"2024-09-05T13:21:53.903975+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:53.903975+0000","last_undegraded":"2024-09-05T13:21:53.903975+0000","last_fullsized":"2024-09-05T13:21:53.903975+0000","mapping_epoch":35,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6320128,"num_objects":1543,"num_object_clones":0,"num_object_copies":3086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19483,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19333","reported_seq":19474,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.782190+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:50.782190+0000","last_peered":"2024-09-05T13:21:50.782190+0000","last_clean":"2024-09-05T13:21:50.782190+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:50.782190+0000","last_undegraded":"2024-09-05T13:21:50.782190+0000","last_fullsized":"2024-09-05T13:21:50.782190+0000","mapping_epoch":34,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6348800,"num_objects":1550,"num_object_clones":0,"num_object_copies":3100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15873,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19583","reported_seq":19727,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.717912+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:50.717912+0000","last_peered":"2024-09-05T13:21:50.717912+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:50.717912+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6438912,"num_objects":1572,"num_object_clones":0,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16038,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19209","reported_seq":28242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:55.087266+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:55.087266+0000","last_peered":"2024-09-05T13:21:55.087266+0000","last_clean":"2024-09-05T13:21:55.087266+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:55.087266+0000","last_undegraded":"2024-09-05T13:21:55.087266+0000","last_fullsized":"2024-09-05T13:21:55.087266+0000","mapping_epoch":46,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5812224,"num_objects":1419,"num_object_clones":0,"num_object_copies":2838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15528,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19552","reported_seq":19690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:50.745159+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:50.745159+0000","last_peered":"2024-09-05T13:21:50.745159+0000","last_clean":"2024-09-05T13:21:50.745159+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:50.745159+0000","last_undegraded":"2024-09-05T13:21:50.745159+0000","last_fullsized":"2024-09-05T13:21:50.745159+0000","mapping_epoch":34,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":1552,"num_object_clones":0,"num_object_copies":3104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16162,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19516","reported_seq":19657,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.834215+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:53.834215+0000","last_peered":"2024-09-05T13:21:53.834215+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:53.834215+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6459392,"num_objects":1577,"num_object_clones":0,"num_object_copies":3154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17170,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2874}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19346","reported_seq":19490,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.703328+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:50.703328+0000","last_peered":"2024-09-05T13:21:50.703328+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:50.703328+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6406144,"num_objects":1564,"num_object_clones":0,"num_object_copies":3128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15713,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2890}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19445","reported_seq":28350,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.904088+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:53.904088+0000","last_peered":"2024-09-05T13:21:53.904088+0000","last_clean":"2024-09-05T13:21:53.904088+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:53.904088+0000","last_undegraded":"2024-09-05T13:21:53.904088+0000","last_fullsized":"2024-09-05T13:21:53.904088+0000","mapping_epoch":44,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6004736,"num_objects":1466,"num_object_clones":0,"num_object_copies":2932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17040,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19090","reported_seq":19219,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.850757+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:53.850757+0000","last_peered":"2024-09-05T13:21:53.850757+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:53.850757+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6111232,"num_objects":1492,"num_object_clones":0,"num_object_copies":2984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16599,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2895}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19319","reported_seq":19464,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.759438+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:50.759438+0000","last_peered":"2024-09-05T13:21:50.759438+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:50.759438+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6385664,"num_objects":1559,"num_object_clones":0,"num_object_copies":3118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15470,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19506","reported_seq":19700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.745092+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:50.745092+0000","last_peered":"2024-09-05T13:21:50.745092+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:50.745092+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15405,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2846}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19518","reported_seq":19663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.780062+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:50.780062+0000","last_peered":"2024-09-05T13:21:50.780062+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:50.780062+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6397952,"num_objects":1562,"num_object_clones":0,"num_object_copies":3124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15648,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3004}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19654","reported_seq":19800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:50.746992+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:50.746992+0000","last_peered":"2024-09-05T13:21:50.746992+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:50.746992+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6300131,"num_objects":1539,"num_object_clones":0,"num_object_copies":3078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16322,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3026}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":101085691,"num_objects":24681,"num_object_clones":0,"num_object_copies":49362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266506,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11288,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417738752,"data_stored":417658348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160688,"ondisk_log_size":160688,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539028,"kb_used_data":192196,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93832812,"statfs":{"total":96636764160,"available":96084799488,"internally_reserved":0,"allocated":196808704,"data_stored":196538702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608072,"kb_used_data":147440,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93763768,"statfs":{"total":96636764160,"available":96014098432,"internally_reserved":0,"allocated":150978560,"data_stored":150708553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,0,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527028,"kb_used_data":71908,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93844812,"statfs":{"total":96636764160,"available":96097087488,"internally_reserved":0,"allocated":73633792,"data_stored":73298292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150024192,"data_stored":150020579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72060928,"data_stored":71991822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195653632,"data_stored":195645947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:57.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:58.285 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:58.310 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-05T13:21:57.730398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":96035342,"num_objects":23450,"num_object_clones":0,"num_object_copies":46900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267741,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11289,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1671428,"kb_used_data":404692,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281444092,"statfs":{"total":289910292480,"available":288198750208,"internally_reserved":0,"allocated":414404608,"data_stored":413529099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1297075206},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,2,6,12,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24391680,"num_objects":-5955,"num_object_clones":0,"num_object_copies":-11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5955,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004348"},"pg_stats":[{"pgid":"2.f","version":"47'19446","reported_seq":19525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.710322+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:55.710322+0000","last_peered":"2024-09-05T13:21:55.710322+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:55.710322+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6250496,"num_objects":1526,"num_object_clones":0,"num_object_copies":3052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19446,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19263","reported_seq":19374,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.908055+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:53.908055+0000","last_peered":"2024-09-05T13:21:53.908055+0000","last_clean":"2024-09-05T13:21:53.908055+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:53.908055+0000","last_undegraded":"2024-09-05T13:21:53.908055+0000","last_fullsized":"2024-09-05T13:21:53.908055+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17273,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19666","reported_seq":24717,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.875978+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:53.875978+0000","last_peered":"2024-09-05T13:21:53.875978+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:53.875978+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262808,"num_objects":1530,"num_object_clones":0,"num_object_copies":3060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1530,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17477,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19483","reported_seq":19559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.903975+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:53.903975+0000","last_peered":"2024-09-05T13:21:53.903975+0000","last_clean":"2024-09-05T13:21:53.903975+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:53.903975+0000","last_undegraded":"2024-09-05T13:21:53.903975+0000","last_fullsized":"2024-09-05T13:21:53.903975+0000","mapping_epoch":35,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6320128,"num_objects":1543,"num_object_clones":0,"num_object_copies":3086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19483,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19476","reported_seq":19617,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:55.760506+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:55.760506+0000","last_peered":"2024-09-05T13:21:55.760506+0000","last_clean":"2024-09-05T13:21:55.760506+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:55.760506+0000","last_undegraded":"2024-09-05T13:21:55.760506+0000","last_fullsized":"2024-09-05T13:21:55.760506+0000","mapping_epoch":34,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5763072,"num_objects":1407,"num_object_clones":0,"num_object_copies":2814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16016,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19709","reported_seq":19854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.756521+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:55.756521+0000","last_peered":"2024-09-05T13:21:55.756521+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:55.756521+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5922816,"num_objects":1446,"num_object_clones":0,"num_object_copies":2892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16164,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19209","reported_seq":28242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:55.087266+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:55.087266+0000","last_peered":"2024-09-05T13:21:55.087266+0000","last_clean":"2024-09-05T13:21:55.087266+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:55.087266+0000","last_undegraded":"2024-09-05T13:21:55.087266+0000","last_fullsized":"2024-09-05T13:21:55.087266+0000","mapping_epoch":46,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5812224,"num_objects":1419,"num_object_clones":0,"num_object_copies":2838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15528,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19700","reported_seq":19838,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:55.774417+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:55.774417+0000","last_peered":"2024-09-05T13:21:55.774417+0000","last_clean":"2024-09-05T13:21:55.774417+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:55.774417+0000","last_undegraded":"2024-09-05T13:21:55.774417+0000","last_fullsized":"2024-09-05T13:21:55.774417+0000","mapping_epoch":34,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5750784,"num_objects":1404,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16310,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19516","reported_seq":19657,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.834215+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:53.834215+0000","last_peered":"2024-09-05T13:21:53.834215+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:53.834215+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6459392,"num_objects":1577,"num_object_clones":0,"num_object_copies":3154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17170,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2874}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19500","reported_seq":19644,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.709738+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:55.709738+0000","last_peered":"2024-09-05T13:21:55.709738+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:55.709738+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5775360,"num_objects":1410,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15867,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3044}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19445","reported_seq":28350,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.904088+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:53.904088+0000","last_peered":"2024-09-05T13:21:53.904088+0000","last_clean":"2024-09-05T13:21:53.904088+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:53.904088+0000","last_undegraded":"2024-09-05T13:21:53.904088+0000","last_fullsized":"2024-09-05T13:21:53.904088+0000","mapping_epoch":44,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6004736,"num_objects":1466,"num_object_clones":0,"num_object_copies":2932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17040,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19090","reported_seq":19219,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.850757+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:53.850757+0000","last_peered":"2024-09-05T13:21:53.850757+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:53.850757+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6111232,"num_objects":1492,"num_object_clones":0,"num_object_copies":2984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16599,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2895}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19438","reported_seq":19585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.793983+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:55.793983+0000","last_peered":"2024-09-05T13:21:55.793983+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:55.793983+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":1440,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15589,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19636","reported_seq":19830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.642327+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:55.642327+0000","last_peered":"2024-09-05T13:21:55.642327+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:55.642327+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6012928,"num_objects":1468,"num_object_clones":0,"num_object_copies":2936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15535,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19649","reported_seq":19794,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.698909+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:55.698909+0000","last_peered":"2024-09-05T13:21:55.698909+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:55.698909+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5861376,"num_objects":1431,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15779,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19795","reported_seq":19940,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.751490+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:55.751490+0000","last_peered":"2024-09-05T13:21:55.751490+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:55.751490+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5722595,"num_objects":1398,"num_object_clones":0,"num_object_copies":2796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16463,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":96035323,"num_objects":23448,"num_object_clones":0,"num_object_copies":46896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23448,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267739,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11288,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":410845184,"data_stored":410764780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160921,"ondisk_log_size":160921,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539028,"kb_used_data":192196,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93832812,"statfs":{"total":96636764160,"available":96084799488,"internally_reserved":0,"allocated":196808704,"data_stored":196538702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605372,"kb_used_data":140588,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93766468,"statfs":{"total":96636764160,"available":96016863232,"internally_reserved":0,"allocated":143962112,"data_stored":143692105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527028,"kb_used_data":71908,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93844812,"statfs":{"total":96636764160,"available":96097087488,"internally_reserved":0,"allocated":73633792,"data_stored":73298292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143130624,"data_stored":143127011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72060928,"data_stored":71991822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195653632,"data_stored":195645947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:21:58.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'19446", 'reported_seq': 19525, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:55.710322+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:21:55.710322+0000', 'last_peered': '2024-09-05T13:21:55.710322+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:21:55.710322+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6250496, 'num_objects': 1526, 'num_object_clones': 0, 'num_object_copies': 3052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19446, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2895}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'19666", 'reported_seq': 24717, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:53.875978+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:53.875978+0000', 'last_peered': '2024-09-05T13:21:53.875978+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:53.875978+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6262808, 'num_objects': 1530, 'num_object_clones': 0, 'num_object_copies': 3060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2035, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1530, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17477, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2458, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -505}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'19709", 'reported_seq': 19854, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:55.756521+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:21:55.756521+0000', 'last_peered': '2024-09-05T13:21:55.756521+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:21:55.756521+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5922816, 'num_objects': 1446, 'num_object_clones': 0, 'num_object_copies': 2892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16164, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3005}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'19516", 'reported_seq': 19657, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:53.834215+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:53.834215+0000', 'last_peered': '2024-09-05T13:21:53.834215+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:53.834215+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6459392, 'num_objects': 1577, 'num_object_clones': 0, 'num_object_copies': 3154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17170, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2874}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'19500", 'reported_seq': 19644, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:55.709738+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:21:55.709738+0000', 'last_peered': '2024-09-05T13:21:55.709738+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:21:55.709738+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5775360, 'num_objects': 1410, 'num_object_clones': 0, 'num_object_copies': 2820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15867, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3044}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:21:58.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'19090", 'reported_seq': 19219, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:53.850757+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:53.850757+0000', 'last_peered': '2024-09-05T13:21:53.850757+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:53.850757+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'9000", 'ondisk_log_start': "43'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6111232, 'num_objects': 1492, 'num_object_clones': 0, 'num_object_copies': 2984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16599, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2895}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:21:58.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:21:58.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'19438", 'reported_seq': 19585, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:55.793983+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:21:55.793983+0000', 'last_peered': '2024-09-05T13:21:55.793983+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:21:55.793983+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'9400", 'ondisk_log_start': "43'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5898240, 'num_objects': 1440, 'num_object_clones': 0, 'num_object_copies': 2880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15589, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2979}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:58.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:21:58.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'19636", 'reported_seq': 19830, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:55.642327+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:21:55.642327+0000', 'last_peered': '2024-09-05T13:21:55.642327+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:21:55.642327+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6012928, 'num_objects': 1468, 'num_object_clones': 0, 'num_object_copies': 2936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1468, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15535, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2976}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:58.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:21:58.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'19649", 'reported_seq': 19794, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:55.698909+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:21:55.698909+0000', 'last_peered': '2024-09-05T13:21:55.698909+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:21:55.698909+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5861376, 'num_objects': 1431, 'num_object_clones': 0, 'num_object_copies': 2862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15779, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3135}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:58.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:21:58.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'19795", 'reported_seq': 19940, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:55.751490+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:21:55.751490+0000', 'last_peered': '2024-09-05T13:21:55.751490+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:21:55.751490+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5722595, 'num_objects': 1398, 'num_object_clones': 0, 'num_object_copies': 2796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1398, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16463, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3167}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:21:58.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:21:58.973 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42210/46900 objects degraded (90.000%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":645,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":23450,"data_bytes":96035342,"bytes_used":1711542272,"bytes_avail":288198750208,"bytes_total":289910292480,"degraded_objects":42210,"degraded_total":46900,"degraded_ratio":0.90000000000000002,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":496},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:21:58.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:21:58.998 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:21:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:21:59.554 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:21:59.579 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-05T13:21:57.730398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":96035342,"num_objects":23450,"num_object_clones":0,"num_object_copies":46900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267741,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11289,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1671428,"kb_used_data":404692,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281444092,"statfs":{"total":289910292480,"available":288198750208,"internally_reserved":0,"allocated":414404608,"data_stored":413529099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1297075206},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,2,6,12,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24391680,"num_objects":-5955,"num_object_clones":0,"num_object_copies":-11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5955,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004348"},"pg_stats":[{"pgid":"2.f","version":"47'19446","reported_seq":19525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.710322+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:21:55.710322+0000","last_peered":"2024-09-05T13:21:55.710322+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:21:55.710322+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6250496,"num_objects":1526,"num_object_clones":0,"num_object_copies":3052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19446,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19263","reported_seq":19374,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.908055+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:53.908055+0000","last_peered":"2024-09-05T13:21:53.908055+0000","last_clean":"2024-09-05T13:21:53.908055+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:53.908055+0000","last_undegraded":"2024-09-05T13:21:53.908055+0000","last_fullsized":"2024-09-05T13:21:53.908055+0000","mapping_epoch":35,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6107136,"num_objects":1491,"num_object_clones":0,"num_object_copies":2982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17273,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19666","reported_seq":24717,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.875978+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:53.875978+0000","last_peered":"2024-09-05T13:21:53.875978+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:53.875978+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262808,"num_objects":1530,"num_object_clones":0,"num_object_copies":3060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1530,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17477,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19483","reported_seq":19559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.903975+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:53.903975+0000","last_peered":"2024-09-05T13:21:53.903975+0000","last_clean":"2024-09-05T13:21:53.903975+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:53.903975+0000","last_undegraded":"2024-09-05T13:21:53.903975+0000","last_fullsized":"2024-09-05T13:21:53.903975+0000","mapping_epoch":35,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6320128,"num_objects":1543,"num_object_clones":0,"num_object_copies":3086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19483,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19476","reported_seq":19617,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:55.760506+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:21:55.760506+0000","last_peered":"2024-09-05T13:21:55.760506+0000","last_clean":"2024-09-05T13:21:55.760506+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:21:55.760506+0000","last_undegraded":"2024-09-05T13:21:55.760506+0000","last_fullsized":"2024-09-05T13:21:55.760506+0000","mapping_epoch":34,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5763072,"num_objects":1407,"num_object_clones":0,"num_object_copies":2814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16016,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19709","reported_seq":19854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.756521+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:21:55.756521+0000","last_peered":"2024-09-05T13:21:55.756521+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:21:55.756521+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5922816,"num_objects":1446,"num_object_clones":0,"num_object_copies":2892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16164,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19209","reported_seq":28242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:55.087266+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:21:55.087266+0000","last_peered":"2024-09-05T13:21:55.087266+0000","last_clean":"2024-09-05T13:21:55.087266+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:21:55.087266+0000","last_undegraded":"2024-09-05T13:21:55.087266+0000","last_fullsized":"2024-09-05T13:21:55.087266+0000","mapping_epoch":46,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5812224,"num_objects":1419,"num_object_clones":0,"num_object_copies":2838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15528,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19700","reported_seq":19838,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:55.774417+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:21:55.774417+0000","last_peered":"2024-09-05T13:21:55.774417+0000","last_clean":"2024-09-05T13:21:55.774417+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:21:55.774417+0000","last_undegraded":"2024-09-05T13:21:55.774417+0000","last_fullsized":"2024-09-05T13:21:55.774417+0000","mapping_epoch":34,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5750784,"num_objects":1404,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16310,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19516","reported_seq":19657,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.834215+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:53.834215+0000","last_peered":"2024-09-05T13:21:53.834215+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:53.834215+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6459392,"num_objects":1577,"num_object_clones":0,"num_object_copies":3154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17170,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2874}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19500","reported_seq":19644,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.709738+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:21:55.709738+0000","last_peered":"2024-09-05T13:21:55.709738+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:21:55.709738+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5775360,"num_objects":1410,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15867,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3044}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19445","reported_seq":28350,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:53.904088+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:53.904088+0000","last_peered":"2024-09-05T13:21:53.904088+0000","last_clean":"2024-09-05T13:21:53.904088+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:53.904088+0000","last_undegraded":"2024-09-05T13:21:53.904088+0000","last_fullsized":"2024-09-05T13:21:53.904088+0000","mapping_epoch":44,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6004736,"num_objects":1466,"num_object_clones":0,"num_object_copies":2932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17040,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19090","reported_seq":19219,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:53.850757+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:53.850757+0000","last_peered":"2024-09-05T13:21:53.850757+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:53.850757+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9000","ondisk_log_start":"43'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6111232,"num_objects":1492,"num_object_clones":0,"num_object_copies":2984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16599,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2895}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19438","reported_seq":19585,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.793983+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:21:55.793983+0000","last_peered":"2024-09-05T13:21:55.793983+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:21:55.793983+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":1440,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15589,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19636","reported_seq":19830,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.642327+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:21:55.642327+0000","last_peered":"2024-09-05T13:21:55.642327+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:21:55.642327+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6012928,"num_objects":1468,"num_object_clones":0,"num_object_copies":2936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15535,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2976}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19649","reported_seq":19794,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.698909+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:21:55.698909+0000","last_peered":"2024-09-05T13:21:55.698909+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:21:55.698909+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5861376,"num_objects":1431,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15779,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.110559+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:36.110559+0000","last_peered":"2024-09-05T13:19:36.110559+0000","last_clean":"2024-09-05T13:19:36.110559+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:36.110559+0000","last_undegraded":"2024-09-05T13:19:36.110559+0000","last_fullsized":"2024-09-05T13:19:36.110559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19795","reported_seq":19940,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:55.751490+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:21:55.751490+0000","last_peered":"2024-09-05T13:21:55.751490+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:21:55.751490+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5722595,"num_objects":1398,"num_object_clones":0,"num_object_copies":2796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16463,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":96035323,"num_objects":23448,"num_object_clones":0,"num_object_copies":46896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23448,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267739,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11288,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":410845184,"data_stored":410764780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160921,"ondisk_log_size":160921,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539028,"kb_used_data":192196,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93832812,"statfs":{"total":96636764160,"available":96084799488,"internally_reserved":0,"allocated":196808704,"data_stored":196538702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605372,"kb_used_data":140588,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93766468,"statfs":{"total":96636764160,"available":96016863232,"internally_reserved":0,"allocated":143962112,"data_stored":143692105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527028,"kb_used_data":71908,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93844812,"statfs":{"total":96636764160,"available":96097087488,"internally_reserved":0,"allocated":73633792,"data_stored":73298292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143130624,"data_stored":143127011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":72060928,"data_stored":71991822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":195653632,"data_stored":195645947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:02.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:03.143 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:03.166 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-05T13:22:01.731835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86409742,"num_objects":21100,"num_object_clones":0,"num_object_copies":42200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270091,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11395,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1640284,"kb_used_data":386008,"kb_used_omap":12,"kb_used_meta":1254195,"kb_avail":281475236,"statfs":{"total":289910292480,"available":288230641664,"internally_reserved":0,"allocated":395272192,"data_stored":394396683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1284295686},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,5,10,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25014272,"num_objects":-6107,"num_object_clones":0,"num_object_copies":-12214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6107,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004205"},"pg_stats":[{"pgid":"2.f","version":"47'19616","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.760931+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:00.760931+0000","last_peered":"2024-09-05T13:22:00.760931+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:00.760931+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5554176,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19616,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3065}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19422","reported_seq":19533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.857250+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:58.857250+0000","last_peered":"2024-09-05T13:21:58.857250+0000","last_clean":"2024-09-05T13:21:58.857250+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:58.857250+0000","last_undegraded":"2024-09-05T13:21:58.857250+0000","last_fullsized":"2024-09-05T13:21:58.857250+0000","mapping_epoch":35,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5455872,"num_objects":1332,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17432,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19827","reported_seq":25091,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.927046+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:58.927046+0000","last_peered":"2024-09-05T13:21:58.927046+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:58.927046+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603352,"num_objects":1369,"num_object_clones":0,"num_object_copies":2738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17638,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2564,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19648","reported_seq":19724,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.841332+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:58.841332+0000","last_peered":"2024-09-05T13:21:58.841332+0000","last_clean":"2024-09-05T13:21:58.841332+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:58.841332+0000","last_undegraded":"2024-09-05T13:21:58.841332+0000","last_fullsized":"2024-09-05T13:21:58.841332+0000","mapping_epoch":35,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5644288,"num_objects":1378,"num_object_clones":0,"num_object_copies":2756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19648,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19617","reported_seq":19758,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.760487+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:00.760487+0000","last_peered":"2024-09-05T13:22:00.760487+0000","last_clean":"2024-09-05T13:22:00.760487+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:00.760487+0000","last_undegraded":"2024-09-05T13:22:00.760487+0000","last_fullsized":"2024-09-05T13:22:00.760487+0000","mapping_epoch":34,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185536,"num_objects":1266,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16157,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19848","reported_seq":19992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.726763+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:00.726763+0000","last_peered":"2024-09-05T13:22:00.726763+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:00.726763+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5353472,"num_objects":1307,"num_object_clones":0,"num_object_copies":2614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16303,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19349","reported_seq":28382,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.210060+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:00.210060+0000","last_peered":"2024-09-05T13:22:00.210060+0000","last_clean":"2024-09-05T13:22:00.210060+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:00.210060+0000","last_undegraded":"2024-09-05T13:22:00.210060+0000","last_fullsized":"2024-09-05T13:22:00.210060+0000","mapping_epoch":46,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5238784,"num_objects":1279,"num_object_clones":0,"num_object_copies":2558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19828","reported_seq":19966,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.810113+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:00.810113+0000","last_peered":"2024-09-05T13:22:00.810113+0000","last_clean":"2024-09-05T13:22:00.810113+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:00.810113+0000","last_undegraded":"2024-09-05T13:22:00.810113+0000","last_fullsized":"2024-09-05T13:22:00.810113+0000","mapping_epoch":34,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226496,"num_objects":1276,"num_object_clones":0,"num_object_copies":2552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16438,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19665","reported_seq":19806,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.887100+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:58.887100+0000","last_peered":"2024-09-05T13:21:58.887100+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:58.887100+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5849088,"num_objects":1428,"num_object_clones":0,"num_object_copies":2856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17319,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19641","reported_seq":19783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.726921+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:00.726921+0000","last_peered":"2024-09-05T13:22:00.726921+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:00.726921+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5197824,"num_objects":1269,"num_object_clones":0,"num_object_copies":2538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16008,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3185}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19580","reported_seq":28487,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.693698+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:58.693698+0000","last_peered":"2024-09-05T13:21:58.693698+0000","last_clean":"2024-09-05T13:21:58.693698+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:58.693698+0000","last_undegraded":"2024-09-05T13:21:58.693698+0000","last_fullsized":"2024-09-05T13:21:58.693698+0000","mapping_epoch":44,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5451776,"num_objects":1331,"num_object_clones":0,"num_object_copies":2662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17175,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19226","reported_seq":19355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.718099+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:58.718099+0000","last_peered":"2024-09-05T13:21:58.718099+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:58.718099+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5554176,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3031}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19594","reported_seq":19735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.745254+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:00.745254+0000","last_peered":"2024-09-05T13:22:00.745254+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:00.745254+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5259264,"num_objects":1284,"num_object_clones":0,"num_object_copies":2568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15745,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19787","reported_seq":19981,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.766053+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:00.766053+0000","last_peered":"2024-09-05T13:22:00.766053+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:00.766053+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5394432,"num_objects":1317,"num_object_clones":0,"num_object_copies":2634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15686,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19791","reported_seq":19936,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.779558+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:00.779558+0000","last_peered":"2024-09-05T13:22:00.779558+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:00.779558+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5279744,"num_objects":1289,"num_object_clones":0,"num_object_copies":2578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15921,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19932","reported_seq":20078,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.807723+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:00.807723+0000","last_peered":"2024-09-05T13:22:00.807723+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:00.807723+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5161443,"num_objects":1261,"num_object_clones":0,"num_object_copies":2522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":86409723,"num_objects":21098,"num_object_clones":0,"num_object_copies":42196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21098,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270089,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11394,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":391684096,"data_stored":391603692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521756,"kb_used_data":187396,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93850084,"statfs":{"total":96636764160,"available":96102486016,"internally_reserved":0,"allocated":191893504,"data_stored":191623502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606736,"kb_used_data":133632,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93765104,"statfs":{"total":96636764160,"available":96015466496,"internally_reserved":0,"allocated":136839168,"data_stored":136569161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511792,"kb_used_data":64980,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93860048,"statfs":{"total":96636764160,"available":96112689152,"internally_reserved":0,"allocated":66539520,"data_stored":66204020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,6,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":135950336,"data_stored":135946723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64942080,"data_stored":64872974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":190791680,"data_stored":190783995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:03.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:03.710 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:03.711 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:03.736 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-05T13:22:01.731835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86409742,"num_objects":21100,"num_object_clones":0,"num_object_copies":42200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270091,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11395,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1640284,"kb_used_data":386008,"kb_used_omap":12,"kb_used_meta":1254195,"kb_avail":281475236,"statfs":{"total":289910292480,"available":288230641664,"internally_reserved":0,"allocated":395272192,"data_stored":394396683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1284295686},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,5,10,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25014272,"num_objects":-6107,"num_object_clones":0,"num_object_copies":-12214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6107,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004205"},"pg_stats":[{"pgid":"2.f","version":"47'19616","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.760931+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:00.760931+0000","last_peered":"2024-09-05T13:22:00.760931+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:00.760931+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5554176,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19616,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3065}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19422","reported_seq":19533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.857250+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:58.857250+0000","last_peered":"2024-09-05T13:21:58.857250+0000","last_clean":"2024-09-05T13:21:58.857250+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:58.857250+0000","last_undegraded":"2024-09-05T13:21:58.857250+0000","last_fullsized":"2024-09-05T13:21:58.857250+0000","mapping_epoch":35,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5455872,"num_objects":1332,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17432,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19827","reported_seq":25091,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.927046+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:58.927046+0000","last_peered":"2024-09-05T13:21:58.927046+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:58.927046+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603352,"num_objects":1369,"num_object_clones":0,"num_object_copies":2738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17638,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2564,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19648","reported_seq":19724,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.841332+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:58.841332+0000","last_peered":"2024-09-05T13:21:58.841332+0000","last_clean":"2024-09-05T13:21:58.841332+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:58.841332+0000","last_undegraded":"2024-09-05T13:21:58.841332+0000","last_fullsized":"2024-09-05T13:21:58.841332+0000","mapping_epoch":35,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5644288,"num_objects":1378,"num_object_clones":0,"num_object_copies":2756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19648,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19617","reported_seq":19758,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.760487+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:00.760487+0000","last_peered":"2024-09-05T13:22:00.760487+0000","last_clean":"2024-09-05T13:22:00.760487+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:00.760487+0000","last_undegraded":"2024-09-05T13:22:00.760487+0000","last_fullsized":"2024-09-05T13:22:00.760487+0000","mapping_epoch":34,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185536,"num_objects":1266,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16157,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19848","reported_seq":19992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.726763+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:00.726763+0000","last_peered":"2024-09-05T13:22:00.726763+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:00.726763+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5353472,"num_objects":1307,"num_object_clones":0,"num_object_copies":2614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16303,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19349","reported_seq":28382,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.210060+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:00.210060+0000","last_peered":"2024-09-05T13:22:00.210060+0000","last_clean":"2024-09-05T13:22:00.210060+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:00.210060+0000","last_undegraded":"2024-09-05T13:22:00.210060+0000","last_fullsized":"2024-09-05T13:22:00.210060+0000","mapping_epoch":46,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5238784,"num_objects":1279,"num_object_clones":0,"num_object_copies":2558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19828","reported_seq":19966,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.810113+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:00.810113+0000","last_peered":"2024-09-05T13:22:00.810113+0000","last_clean":"2024-09-05T13:22:00.810113+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:00.810113+0000","last_undegraded":"2024-09-05T13:22:00.810113+0000","last_fullsized":"2024-09-05T13:22:00.810113+0000","mapping_epoch":34,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226496,"num_objects":1276,"num_object_clones":0,"num_object_copies":2552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16438,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19665","reported_seq":19806,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.887100+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:58.887100+0000","last_peered":"2024-09-05T13:21:58.887100+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:58.887100+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5849088,"num_objects":1428,"num_object_clones":0,"num_object_copies":2856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17319,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19641","reported_seq":19783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.726921+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:00.726921+0000","last_peered":"2024-09-05T13:22:00.726921+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:00.726921+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5197824,"num_objects":1269,"num_object_clones":0,"num_object_copies":2538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16008,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3185}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19580","reported_seq":28487,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.693698+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:58.693698+0000","last_peered":"2024-09-05T13:21:58.693698+0000","last_clean":"2024-09-05T13:21:58.693698+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:58.693698+0000","last_undegraded":"2024-09-05T13:21:58.693698+0000","last_fullsized":"2024-09-05T13:21:58.693698+0000","mapping_epoch":44,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5451776,"num_objects":1331,"num_object_clones":0,"num_object_copies":2662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17175,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19226","reported_seq":19355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.718099+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:58.718099+0000","last_peered":"2024-09-05T13:21:58.718099+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:58.718099+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5554176,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3031}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19594","reported_seq":19735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.745254+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:00.745254+0000","last_peered":"2024-09-05T13:22:00.745254+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:00.745254+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5259264,"num_objects":1284,"num_object_clones":0,"num_object_copies":2568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15745,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19787","reported_seq":19981,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.766053+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:00.766053+0000","last_peered":"2024-09-05T13:22:00.766053+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:00.766053+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5394432,"num_objects":1317,"num_object_clones":0,"num_object_copies":2634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15686,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19791","reported_seq":19936,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.779558+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:00.779558+0000","last_peered":"2024-09-05T13:22:00.779558+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:00.779558+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5279744,"num_objects":1289,"num_object_clones":0,"num_object_copies":2578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15921,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19932","reported_seq":20078,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.807723+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:00.807723+0000","last_peered":"2024-09-05T13:22:00.807723+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:00.807723+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5161443,"num_objects":1261,"num_object_clones":0,"num_object_copies":2522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":86409723,"num_objects":21098,"num_object_clones":0,"num_object_copies":42196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21098,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270089,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11394,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":391684096,"data_stored":391603692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521756,"kb_used_data":187396,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93850084,"statfs":{"total":96636764160,"available":96102486016,"internally_reserved":0,"allocated":191893504,"data_stored":191623502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606736,"kb_used_data":133632,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93765104,"statfs":{"total":96636764160,"available":96015466496,"internally_reserved":0,"allocated":136839168,"data_stored":136569161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511792,"kb_used_data":64980,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93860048,"statfs":{"total":96636764160,"available":96112689152,"internally_reserved":0,"allocated":66539520,"data_stored":66204020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,6,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":135950336,"data_stored":135946723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64942080,"data_stored":64872974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":190791680,"data_stored":190783995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:03.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:03.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'19616", 'reported_seq': 19692, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:00.760931+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:00.760931+0000', 'last_peered': '2024-09-05T13:22:00.760931+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:00.760931+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5554176, 'num_objects': 1356, 'num_object_clones': 0, 'num_object_copies': 2712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19616, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3065}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:03.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:03.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'19827", 'reported_seq': 25091, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:58.927046+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:21:58.927046+0000', 'last_peered': '2024-09-05T13:21:58.927046+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:21:58.927046+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5603352, 'num_objects': 1369, 'num_object_clones': 0, 'num_object_copies': 2738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1929, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1369, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17638, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2564, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -560}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'19848", 'reported_seq': 19992, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:00.726763+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:00.726763+0000', 'last_peered': '2024-09-05T13:22:00.726763+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:00.726763+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5353472, 'num_objects': 1307, 'num_object_clones': 0, 'num_object_copies': 2614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16303, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'19665", 'reported_seq': 19806, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:58.887100+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:21:58.887100+0000', 'last_peered': '2024-09-05T13:21:58.887100+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:21:58.887100+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5849088, 'num_objects': 1428, 'num_object_clones': 0, 'num_object_copies': 2856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17319, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3023}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'19641", 'reported_seq': 19783, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:00.726921+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:00.726921+0000', 'last_peered': '2024-09-05T13:22:00.726921+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:00.726921+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'9600", 'ondisk_log_start': "43'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5197824, 'num_objects': 1269, 'num_object_clones': 0, 'num_object_copies': 2538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1269, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16008, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3185}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'19226", 'reported_seq': 19355, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:21:58.718099+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:21:58.718099+0000', 'last_peered': '2024-09-05T13:21:58.718099+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:21:58.718099+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'9200", 'ondisk_log_start': "43'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5554176, 'num_objects': 1356, 'num_object_clones': 0, 'num_object_copies': 2712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16735, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3031}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:03.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'19594", 'reported_seq': 19735, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:00.745254+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:00.745254+0000', 'last_peered': '2024-09-05T13:22:00.745254+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:00.745254+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5259264, 'num_objects': 1284, 'num_object_clones': 0, 'num_object_copies': 2568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1284, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15745, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3135}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:03.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:03.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'19787", 'reported_seq': 19981, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:00.766053+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:00.766053+0000', 'last_peered': '2024-09-05T13:22:00.766053+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:00.766053+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5394432, 'num_objects': 1317, 'num_object_clones': 0, 'num_object_copies': 2634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1317, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15686, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3127}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:03.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:03.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'19791", 'reported_seq': 19936, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:00.779558+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:00.779558+0000', 'last_peered': '2024-09-05T13:22:00.779558+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:00.779558+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5279744, 'num_objects': 1289, 'num_object_clones': 0, 'num_object_copies': 2578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15921, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3277}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:03.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:03.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'19932", 'reported_seq': 20078, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:00.807723+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:00.807723+0000', 'last_peered': '2024-09-05T13:22:00.807723+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:00.807723+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5161443, 'num_objects': 1261, 'num_object_clones': 0, 'num_object_copies': 2522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16600, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:03.739 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:04.399 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:04.399 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 42104/42200 objects degraded (99.773%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":651,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":21100,"data_bytes":86409742,"bytes_used":1679650816,"bytes_avail":288230641664,"bytes_total":289910292480,"degraded_objects":42104,"degraded_total":42200,"degraded_ratio":0.99772511848341228,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:04.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:04.428 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:04.982 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:04.983 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:05.003 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-05T13:22:03.732678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":86409742,"num_objects":21100,"num_object_clones":0,"num_object_copies":42200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270091,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11395,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1640284,"kb_used_data":386008,"kb_used_omap":12,"kb_used_meta":1254195,"kb_avail":281475236,"statfs":{"total":289910292480,"available":288230641664,"internally_reserved":0,"allocated":395272192,"data_stored":394396683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1284295686},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,5,10,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19050496,"num_objects":-4651,"num_object_clones":0,"num_object_copies":-9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4651,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004284"},"pg_stats":[{"pgid":"2.f","version":"47'19616","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.760931+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:00.760931+0000","last_peered":"2024-09-05T13:22:00.760931+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:00.760931+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5554176,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19616,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3065}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19422","reported_seq":19533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.857250+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:21:58.857250+0000","last_peered":"2024-09-05T13:21:58.857250+0000","last_clean":"2024-09-05T13:21:58.857250+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:21:58.857250+0000","last_undegraded":"2024-09-05T13:21:58.857250+0000","last_fullsized":"2024-09-05T13:21:58.857250+0000","mapping_epoch":35,"log_start":"43'9400","ondisk_log_start":"43'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5455872,"num_objects":1332,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17432,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19827","reported_seq":25091,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.927046+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:21:58.927046+0000","last_peered":"2024-09-05T13:21:58.927046+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:21:58.927046+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603352,"num_objects":1369,"num_object_clones":0,"num_object_copies":2738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17638,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2564,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19648","reported_seq":19724,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.841332+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:21:58.841332+0000","last_peered":"2024-09-05T13:21:58.841332+0000","last_clean":"2024-09-05T13:21:58.841332+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:21:58.841332+0000","last_undegraded":"2024-09-05T13:21:58.841332+0000","last_fullsized":"2024-09-05T13:21:58.841332+0000","mapping_epoch":35,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5644288,"num_objects":1378,"num_object_clones":0,"num_object_copies":2756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19648,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19617","reported_seq":19758,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.760487+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:00.760487+0000","last_peered":"2024-09-05T13:22:00.760487+0000","last_clean":"2024-09-05T13:22:00.760487+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:00.760487+0000","last_undegraded":"2024-09-05T13:22:00.760487+0000","last_fullsized":"2024-09-05T13:22:00.760487+0000","mapping_epoch":34,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5185536,"num_objects":1266,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16157,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19848","reported_seq":19992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.726763+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:00.726763+0000","last_peered":"2024-09-05T13:22:00.726763+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:00.726763+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5353472,"num_objects":1307,"num_object_clones":0,"num_object_copies":2614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16303,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19349","reported_seq":28382,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.210060+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:00.210060+0000","last_peered":"2024-09-05T13:22:00.210060+0000","last_clean":"2024-09-05T13:22:00.210060+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:00.210060+0000","last_undegraded":"2024-09-05T13:22:00.210060+0000","last_fullsized":"2024-09-05T13:22:00.210060+0000","mapping_epoch":46,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5238784,"num_objects":1279,"num_object_clones":0,"num_object_copies":2558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19828","reported_seq":19966,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:00.810113+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:00.810113+0000","last_peered":"2024-09-05T13:22:00.810113+0000","last_clean":"2024-09-05T13:22:00.810113+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:00.810113+0000","last_undegraded":"2024-09-05T13:22:00.810113+0000","last_fullsized":"2024-09-05T13:22:00.810113+0000","mapping_epoch":34,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226496,"num_objects":1276,"num_object_clones":0,"num_object_copies":2552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16438,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19665","reported_seq":19806,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.887100+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:21:58.887100+0000","last_peered":"2024-09-05T13:21:58.887100+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:21:58.887100+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5849088,"num_objects":1428,"num_object_clones":0,"num_object_copies":2856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17319,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3023}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19641","reported_seq":19783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.726921+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:00.726921+0000","last_peered":"2024-09-05T13:22:00.726921+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:00.726921+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5197824,"num_objects":1269,"num_object_clones":0,"num_object_copies":2538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16008,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3185}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19580","reported_seq":28487,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:21:58.693698+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:21:58.693698+0000","last_peered":"2024-09-05T13:21:58.693698+0000","last_clean":"2024-09-05T13:21:58.693698+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:21:58.693698+0000","last_undegraded":"2024-09-05T13:21:58.693698+0000","last_fullsized":"2024-09-05T13:21:58.693698+0000","mapping_epoch":44,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5451776,"num_objects":1331,"num_object_clones":0,"num_object_copies":2662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17175,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19226","reported_seq":19355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:21:58.718099+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:21:58.718099+0000","last_peered":"2024-09-05T13:21:58.718099+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:21:58.718099+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9200","ondisk_log_start":"43'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5554176,"num_objects":1356,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3031}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19594","reported_seq":19735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.745254+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:00.745254+0000","last_peered":"2024-09-05T13:22:00.745254+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:00.745254+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5259264,"num_objects":1284,"num_object_clones":0,"num_object_copies":2568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15745,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19787","reported_seq":19981,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.766053+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:00.766053+0000","last_peered":"2024-09-05T13:22:00.766053+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:00.766053+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5394432,"num_objects":1317,"num_object_clones":0,"num_object_copies":2634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15686,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19791","reported_seq":19936,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.779558+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:00.779558+0000","last_peered":"2024-09-05T13:22:00.779558+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:00.779558+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5279744,"num_objects":1289,"num_object_clones":0,"num_object_copies":2578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15921,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19932","reported_seq":20078,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:00.807723+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:00.807723+0000","last_peered":"2024-09-05T13:22:00.807723+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:00.807723+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5161443,"num_objects":1261,"num_object_clones":0,"num_object_copies":2522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":86409723,"num_objects":21098,"num_object_clones":0,"num_object_copies":42196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21098,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270089,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11394,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":391684096,"data_stored":391603692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521756,"kb_used_data":187396,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93850084,"statfs":{"total":96636764160,"available":96102486016,"internally_reserved":0,"allocated":191893504,"data_stored":191623502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606736,"kb_used_data":133632,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93765104,"statfs":{"total":96636764160,"available":96015466496,"internally_reserved":0,"allocated":136839168,"data_stored":136569161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511792,"kb_used_data":64980,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93860048,"statfs":{"total":96636764160,"available":96112689152,"internally_reserved":0,"allocated":66539520,"data_stored":66204020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,6,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":135950336,"data_stored":135946723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64942080,"data_stored":64872974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":190791680,"data_stored":190783995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:08.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:08.578 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:08.605 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-05T13:22:07.734131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":76952078,"num_objects":18791,"num_object_clones":0,"num_object_copies":37582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18791,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272400,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11503,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1617596,"kb_used_data":367492,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281497924,"statfs":{"total":289910292480,"available":288253874176,"internally_reserved":0,"allocated":376311808,"data_stored":375436299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1280035846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,6,13,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24133632,"num_objects":-5892,"num_object_clones":0,"num_object_copies":-11784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004388"},"pg_stats":[{"pgid":"2.f","version":"47'19759","reported_seq":19840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.804468+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:05.804468+0000","last_peered":"2024-09-05T13:22:05.804468+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:05.804468+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4968448,"num_objects":1213,"num_object_clones":0,"num_object_copies":2426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19759,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3208}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19559","reported_seq":19670,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:03.847692+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:03.847692+0000","last_peered":"2024-09-05T13:22:03.847692+0000","last_clean":"2024-09-05T13:22:03.847692+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:03.847692+0000","last_undegraded":"2024-09-05T13:22:03.847692+0000","last_fullsized":"2024-09-05T13:22:03.847692+0000","mapping_epoch":35,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4894720,"num_objects":1195,"num_object_clones":0,"num_object_copies":2390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17569,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19973","reported_seq":25452,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:03.824861+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:03.824861+0000","last_peered":"2024-09-05T13:22:03.824861+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:03.824861+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5005336,"num_objects":1223,"num_object_clones":0,"num_object_copies":2446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17784,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2672,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19812","reported_seq":19888,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:03.790552+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:03.790552+0000","last_peered":"2024-09-05T13:22:03.790552+0000","last_clean":"2024-09-05T13:22:03.790552+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:03.790552+0000","last_undegraded":"2024-09-05T13:22:03.790552+0000","last_fullsized":"2024-09-05T13:22:03.790552+0000","mapping_epoch":35,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4972544,"num_objects":1214,"num_object_clones":0,"num_object_copies":2428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19812,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19768","reported_seq":19909,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.717107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:05.717107+0000","last_peered":"2024-09-05T13:22:05.717107+0000","last_clean":"2024-09-05T13:22:05.717107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:05.717107+0000","last_undegraded":"2024-09-05T13:22:05.717107+0000","last_fullsized":"2024-09-05T13:22:05.717107+0000","mapping_epoch":34,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4567040,"num_objects":1115,"num_object_clones":0,"num_object_copies":2230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16308,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19990","reported_seq":20135,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.856597+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:05.856597+0000","last_peered":"2024-09-05T13:22:05.856597+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:05.856597+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4771840,"num_objects":1165,"num_object_clones":0,"num_object_copies":2330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16445,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19501","reported_seq":28534,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.198060+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:05.198060+0000","last_peered":"2024-09-05T13:22:05.198060+0000","last_clean":"2024-09-05T13:22:05.198060+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:05.198060+0000","last_undegraded":"2024-09-05T13:22:05.198060+0000","last_fullsized":"2024-09-05T13:22:05.198060+0000","mapping_epoch":46,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4616192,"num_objects":1127,"num_object_clones":0,"num_object_copies":2254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15820,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19973","reported_seq":20111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.837811+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:05.837811+0000","last_peered":"2024-09-05T13:22:05.837811+0000","last_clean":"2024-09-05T13:22:05.837811+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:05.837811+0000","last_undegraded":"2024-09-05T13:22:05.837811+0000","last_fullsized":"2024-09-05T13:22:05.837811+0000","mapping_epoch":34,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632576,"num_objects":1131,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16583,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19817","reported_seq":19955,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:03.855974+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:03.855974+0000","last_peered":"2024-09-05T13:22:03.855974+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:03.855974+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226496,"num_objects":1276,"num_object_clones":0,"num_object_copies":2552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17471,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19780","reported_seq":19924,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.823669+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:05.823669+0000","last_peered":"2024-09-05T13:22:05.823669+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:05.823669+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4628480,"num_objects":1130,"num_object_clones":0,"num_object_copies":2260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3324}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19730","reported_seq":28637,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:03.889542+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:03.889542+0000","last_peered":"2024-09-05T13:22:03.889542+0000","last_clean":"2024-09-05T13:22:03.889542+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:03.889542+0000","last_undegraded":"2024-09-05T13:22:03.889542+0000","last_fullsized":"2024-09-05T13:22:03.889542+0000","mapping_epoch":44,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4837376,"num_objects":1181,"num_object_clones":0,"num_object_copies":2362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17325,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19374","reported_seq":19503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:03.895504+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:03.895504+0000","last_peered":"2024-09-05T13:22:03.895504+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:03.895504+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4947968,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16883,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3179}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19724","reported_seq":19871,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.799889+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:05.799889+0000","last_peered":"2024-09-05T13:22:05.799889+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:05.799889+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4726784,"num_objects":1154,"num_object_clones":0,"num_object_copies":2308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15875,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19929","reported_seq":20123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.836862+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:05.836862+0000","last_peered":"2024-09-05T13:22:05.836862+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:05.836862+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"44'9900","ondisk_log_start":"44'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4812800,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19935","reported_seq":20080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.710985+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:05.710985+0000","last_peered":"2024-09-05T13:22:05.710985+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:05.710985+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4689920,"num_objects":1145,"num_object_clones":0,"num_object_copies":2290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16065,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20056","reported_seq":20202,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.835398+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:05.835398+0000","last_peered":"2024-09-05T13:22:05.835398+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:05.835398+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653539,"num_objects":1137,"num_object_clones":0,"num_object_copies":2274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16724,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":76952059,"num_objects":18789,"num_object_clones":0,"num_object_copies":37578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272398,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11502,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372695040,"data_stored":372614636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521188,"kb_used_data":182676,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93850652,"statfs":{"total":96636764160,"available":96103067648,"internally_reserved":0,"allocated":187060224,"data_stored":186790222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587332,"kb_used_data":126700,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93784508,"statfs":{"total":96636764160,"available":96035336192,"internally_reserved":0,"allocated":129740800,"data_stored":129470793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509076,"kb_used_data":58116,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93862764,"statfs":{"total":96636764160,"available":96115470336,"internally_reserved":0,"allocated":59510784,"data_stored":59175284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128843776,"data_stored":128840163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57876480,"data_stored":57807374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":185974784,"data_stored":185967099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:08.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:09.148 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:09.175 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-05T13:22:07.734131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":76952078,"num_objects":18791,"num_object_clones":0,"num_object_copies":37582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18791,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272400,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11503,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1617596,"kb_used_data":367492,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281497924,"statfs":{"total":289910292480,"available":288253874176,"internally_reserved":0,"allocated":376311808,"data_stored":375436299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1280035846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,6,13,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24133632,"num_objects":-5892,"num_object_clones":0,"num_object_copies":-11784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004388"},"pg_stats":[{"pgid":"2.f","version":"47'19759","reported_seq":19840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.804468+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:05.804468+0000","last_peered":"2024-09-05T13:22:05.804468+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:05.804468+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4968448,"num_objects":1213,"num_object_clones":0,"num_object_copies":2426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19759,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3208}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19559","reported_seq":19670,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:03.847692+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:03.847692+0000","last_peered":"2024-09-05T13:22:03.847692+0000","last_clean":"2024-09-05T13:22:03.847692+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:03.847692+0000","last_undegraded":"2024-09-05T13:22:03.847692+0000","last_fullsized":"2024-09-05T13:22:03.847692+0000","mapping_epoch":35,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4894720,"num_objects":1195,"num_object_clones":0,"num_object_copies":2390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17569,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19973","reported_seq":25452,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:03.824861+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:03.824861+0000","last_peered":"2024-09-05T13:22:03.824861+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:03.824861+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5005336,"num_objects":1223,"num_object_clones":0,"num_object_copies":2446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1821,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17784,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2672,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-598}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19812","reported_seq":19888,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:03.790552+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:03.790552+0000","last_peered":"2024-09-05T13:22:03.790552+0000","last_clean":"2024-09-05T13:22:03.790552+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:03.790552+0000","last_undegraded":"2024-09-05T13:22:03.790552+0000","last_fullsized":"2024-09-05T13:22:03.790552+0000","mapping_epoch":35,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4972544,"num_objects":1214,"num_object_clones":0,"num_object_copies":2428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19812,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19768","reported_seq":19909,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.717107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:05.717107+0000","last_peered":"2024-09-05T13:22:05.717107+0000","last_clean":"2024-09-05T13:22:05.717107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:05.717107+0000","last_undegraded":"2024-09-05T13:22:05.717107+0000","last_fullsized":"2024-09-05T13:22:05.717107+0000","mapping_epoch":34,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4567040,"num_objects":1115,"num_object_clones":0,"num_object_copies":2230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16308,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19990","reported_seq":20135,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.856597+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:05.856597+0000","last_peered":"2024-09-05T13:22:05.856597+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:05.856597+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4771840,"num_objects":1165,"num_object_clones":0,"num_object_copies":2330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16445,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19501","reported_seq":28534,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.198060+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:05.198060+0000","last_peered":"2024-09-05T13:22:05.198060+0000","last_clean":"2024-09-05T13:22:05.198060+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:05.198060+0000","last_undegraded":"2024-09-05T13:22:05.198060+0000","last_fullsized":"2024-09-05T13:22:05.198060+0000","mapping_epoch":46,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4616192,"num_objects":1127,"num_object_clones":0,"num_object_copies":2254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15820,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19973","reported_seq":20111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.837811+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:05.837811+0000","last_peered":"2024-09-05T13:22:05.837811+0000","last_clean":"2024-09-05T13:22:05.837811+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:05.837811+0000","last_undegraded":"2024-09-05T13:22:05.837811+0000","last_fullsized":"2024-09-05T13:22:05.837811+0000","mapping_epoch":34,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632576,"num_objects":1131,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16583,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19817","reported_seq":19955,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:03.855974+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:03.855974+0000","last_peered":"2024-09-05T13:22:03.855974+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:03.855974+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226496,"num_objects":1276,"num_object_clones":0,"num_object_copies":2552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17471,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19780","reported_seq":19924,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.823669+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:05.823669+0000","last_peered":"2024-09-05T13:22:05.823669+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:05.823669+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4628480,"num_objects":1130,"num_object_clones":0,"num_object_copies":2260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3324}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19730","reported_seq":28637,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:03.889542+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:03.889542+0000","last_peered":"2024-09-05T13:22:03.889542+0000","last_clean":"2024-09-05T13:22:03.889542+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:03.889542+0000","last_undegraded":"2024-09-05T13:22:03.889542+0000","last_fullsized":"2024-09-05T13:22:03.889542+0000","mapping_epoch":44,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4837376,"num_objects":1181,"num_object_clones":0,"num_object_copies":2362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17325,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19374","reported_seq":19503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:03.895504+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:03.895504+0000","last_peered":"2024-09-05T13:22:03.895504+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:03.895504+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9300","ondisk_log_start":"43'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4947968,"num_objects":1208,"num_object_clones":0,"num_object_copies":2416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16883,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3179}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19724","reported_seq":19871,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.799889+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:05.799889+0000","last_peered":"2024-09-05T13:22:05.799889+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:05.799889+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4726784,"num_objects":1154,"num_object_clones":0,"num_object_copies":2308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15875,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19929","reported_seq":20123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.836862+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:05.836862+0000","last_peered":"2024-09-05T13:22:05.836862+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:05.836862+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"44'9900","ondisk_log_start":"44'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4812800,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19935","reported_seq":20080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.710985+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:05.710985+0000","last_peered":"2024-09-05T13:22:05.710985+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:05.710985+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4689920,"num_objects":1145,"num_object_clones":0,"num_object_copies":2290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16065,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20056","reported_seq":20202,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.835398+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:05.835398+0000","last_peered":"2024-09-05T13:22:05.835398+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:05.835398+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653539,"num_objects":1137,"num_object_clones":0,"num_object_copies":2274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16724,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":76952059,"num_objects":18789,"num_object_clones":0,"num_object_copies":37578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41996,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272398,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11502,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372695040,"data_stored":372614636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521188,"kb_used_data":182676,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93850652,"statfs":{"total":96636764160,"available":96103067648,"internally_reserved":0,"allocated":187060224,"data_stored":186790222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587332,"kb_used_data":126700,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93784508,"statfs":{"total":96636764160,"available":96035336192,"internally_reserved":0,"allocated":129740800,"data_stored":129470793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509076,"kb_used_data":58116,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93862764,"statfs":{"total":96636764160,"available":96115470336,"internally_reserved":0,"allocated":59510784,"data_stored":59175284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128843776,"data_stored":128840163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57876480,"data_stored":57807374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":185974784,"data_stored":185967099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:09.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:09.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'19759", 'reported_seq': 19840, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:05.804468+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:05.804468+0000', 'last_peered': '2024-09-05T13:22:05.804468+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:05.804468+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4968448, 'num_objects': 1213, 'num_object_clones': 0, 'num_object_copies': 2426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1213, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19759, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3208}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:09.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:09.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'19973", 'reported_seq': 25452, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:03.824861+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:22:03.824861+0000', 'last_peered': '2024-09-05T13:22:03.824861+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:03.824861+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5005336, 'num_objects': 1223, 'num_object_clones': 0, 'num_object_copies': 2446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1821, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1223, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17784, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2672, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:09.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:09.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'19990", 'reported_seq': 20135, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:05.856597+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:05.856597+0000', 'last_peered': '2024-09-05T13:22:05.856597+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:05.856597+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "43'9900", 'ondisk_log_start': "43'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4771840, 'num_objects': 1165, 'num_object_clones': 0, 'num_object_copies': 2330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16445, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3286}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'19817", 'reported_seq': 19955, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:03.855974+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:03.855974+0000', 'last_peered': '2024-09-05T13:22:03.855974+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:03.855974+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "43'9800", 'ondisk_log_start': "43'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5226496, 'num_objects': 1276, 'num_object_clones': 0, 'num_object_copies': 2552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1276, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17471, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3175}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'19780", 'reported_seq': 19924, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:05.823669+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:05.823669+0000', 'last_peered': '2024-09-05T13:22:05.823669+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:05.823669+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4628480, 'num_objects': 1130, 'num_object_clones': 0, 'num_object_copies': 2260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16147, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3324}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'19374", 'reported_seq': 19503, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:03.895504+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:03.895504+0000', 'last_peered': '2024-09-05T13:22:03.895504+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:03.895504+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'9300", 'ondisk_log_start': "43'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4947968, 'num_objects': 1208, 'num_object_clones': 0, 'num_object_copies': 2416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1208, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16883, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3179}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'19724", 'reported_seq': 19871, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:05.799889+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:05.799889+0000', 'last_peered': '2024-09-05T13:22:05.799889+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:05.799889+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "43'9700", 'ondisk_log_start': "43'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4726784, 'num_objects': 1154, 'num_object_clones': 0, 'num_object_copies': 2308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15875, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3265}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:09.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'19929", 'reported_seq': 20123, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:05.836862+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:05.836862+0000', 'last_peered': '2024-09-05T13:22:05.836862+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:05.836862+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "44'9900", 'ondisk_log_start': "44'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4812800, 'num_objects': 1175, 'num_object_clones': 0, 'num_object_copies': 2350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15828, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3269}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:09.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:09.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'19935", 'reported_seq': 20080, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:05.710985+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:05.710985+0000', 'last_peered': '2024-09-05T13:22:05.710985+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:05.710985+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4689920, 'num_objects': 1145, 'num_object_clones': 0, 'num_object_copies': 2290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16065, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:09.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:09.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'20056", 'reported_seq': 20202, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:05.835398+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:05.835398+0000', 'last_peered': '2024-09-05T13:22:05.835398+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:05.835398+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4653539, 'num_objects': 1137, 'num_object_clones': 0, 'num_object_copies': 2274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1137, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16724, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:09.178 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:09.864 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:09.865 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41996/37582 objects degraded (111.745%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":656,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":18791,"data_bytes":76952078,"bytes_used":1656418304,"bytes_avail":288253874176,"bytes_total":289910292480,"degraded_objects":41996,"degraded_total":37582,"degraded_ratio":1.1174498430099515,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":214,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":490},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:09.888 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:09.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:10.448 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-05T13:22:09.734747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":73282062,"num_objects":17895,"num_object_clones":0,"num_object_copies":35790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41886,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273296,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11613,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1617004,"kb_used_data":362740,"kb_used_omap":12,"kb_used_meta":1254195,"kb_avail":281498516,"statfs":{"total":289910292480,"available":288254480384,"internally_reserved":0,"allocated":371445760,"data_stored":370570251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1284295686},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,6,13,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22753280,"num_objects":-5555,"num_object_clones":0,"num_object_copies":-11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5555,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004348"},"pg_stats":[{"pgid":"2.f","version":"47'19759","reported_seq":19840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.804468+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:05.804468+0000","last_peered":"2024-09-05T13:22:05.804468+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:05.804468+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4968448,"num_objects":1213,"num_object_clones":0,"num_object_copies":2426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19759,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3208}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19710","reported_seq":19821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.937831+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:08.937831+0000","last_peered":"2024-09-05T13:22:08.937831+0000","last_clean":"2024-09-05T13:22:08.937831+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:08.937831+0000","last_undegraded":"2024-09-05T13:22:08.937831+0000","last_fullsized":"2024-09-05T13:22:08.937831+0000","mapping_epoch":35,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4276224,"num_objects":1044,"num_object_clones":0,"num_object_copies":2088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17720,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20123","reported_seq":25819,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.948304+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:08.948304+0000","last_peered":"2024-09-05T13:22:08.948304+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:08.948304+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390936,"num_objects":1073,"num_object_clones":0,"num_object_copies":2146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1073,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17934,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2782,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19960","reported_seq":20036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.844265+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:08.844265+0000","last_peered":"2024-09-05T13:22:08.844265+0000","last_clean":"2024-09-05T13:22:08.844265+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:08.844265+0000","last_undegraded":"2024-09-05T13:22:08.844265+0000","last_fullsized":"2024-09-05T13:22:08.844265+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4366336,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19768","reported_seq":19909,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.717107+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:05.717107+0000","last_peered":"2024-09-05T13:22:05.717107+0000","last_clean":"2024-09-05T13:22:05.717107+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:05.717107+0000","last_undegraded":"2024-09-05T13:22:05.717107+0000","last_fullsized":"2024-09-05T13:22:05.717107+0000","mapping_epoch":34,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4567040,"num_objects":1115,"num_object_clones":0,"num_object_copies":2230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16308,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19990","reported_seq":20135,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.856597+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:05.856597+0000","last_peered":"2024-09-05T13:22:05.856597+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:05.856597+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4771840,"num_objects":1165,"num_object_clones":0,"num_object_copies":2330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16445,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3286}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19501","reported_seq":28534,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.198060+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:05.198060+0000","last_peered":"2024-09-05T13:22:05.198060+0000","last_clean":"2024-09-05T13:22:05.198060+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:05.198060+0000","last_undegraded":"2024-09-05T13:22:05.198060+0000","last_fullsized":"2024-09-05T13:22:05.198060+0000","mapping_epoch":46,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4616192,"num_objects":1127,"num_object_clones":0,"num_object_copies":2254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15820,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'19973","reported_seq":20111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:05.837811+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:05.837811+0000","last_peered":"2024-09-05T13:22:05.837811+0000","last_clean":"2024-09-05T13:22:05.837811+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:05.837811+0000","last_undegraded":"2024-09-05T13:22:05.837811+0000","last_fullsized":"2024-09-05T13:22:05.837811+0000","mapping_epoch":34,"log_start":"43'9900","ondisk_log_start":"43'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632576,"num_objects":1131,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16583,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19972","reported_seq":20113,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.891694+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:08.891694+0000","last_peered":"2024-09-05T13:22:08.891694+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:08.891694+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4591616,"num_objects":1121,"num_object_clones":0,"num_object_copies":2242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17626,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3330}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19780","reported_seq":19924,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.823669+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:05.823669+0000","last_peered":"2024-09-05T13:22:05.823669+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:05.823669+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4628480,"num_objects":1130,"num_object_clones":0,"num_object_copies":2260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3324}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19870","reported_seq":28777,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.930129+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:08.930129+0000","last_peered":"2024-09-05T13:22:08.930129+0000","last_clean":"2024-09-05T13:22:08.930129+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:08.930129+0000","last_undegraded":"2024-09-05T13:22:08.930129+0000","last_fullsized":"2024-09-05T13:22:08.930129+0000","mapping_epoch":44,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4263936,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17465,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19526","reported_seq":19650,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.844798+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:08.844798+0000","last_peered":"2024-09-05T13:22:08.844798+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:08.844798+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":1056,"num_object_clones":0,"num_object_copies":2112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17035,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19724","reported_seq":19871,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.799889+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:05.799889+0000","last_peered":"2024-09-05T13:22:05.799889+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:05.799889+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4726784,"num_objects":1154,"num_object_clones":0,"num_object_copies":2308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15875,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19929","reported_seq":20123,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.836862+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:05.836862+0000","last_peered":"2024-09-05T13:22:05.836862+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:05.836862+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"44'9900","ondisk_log_start":"44'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4812800,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19935","reported_seq":20080,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.710985+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:05.710985+0000","last_peered":"2024-09-05T13:22:05.710985+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:05.710985+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4689920,"num_objects":1145,"num_object_clones":0,"num_object_copies":2290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16065,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20056","reported_seq":20202,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:05.835398+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:05.835398+0000","last_peered":"2024-09-05T13:22:05.835398+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:05.835398+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653539,"num_objects":1137,"num_object_clones":0,"num_object_copies":2274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16724,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":73282043,"num_objects":17893,"num_object_clones":0,"num_object_copies":35786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41886,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":273294,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11612,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":367874048,"data_stored":367793644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520596,"kb_used_data":177924,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93851244,"statfs":{"total":96636764160,"available":96103673856,"internally_reserved":0,"allocated":182194176,"data_stored":181924174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587332,"kb_used_data":126700,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93784508,"statfs":{"total":96636764160,"available":96035336192,"internally_reserved":0,"allocated":129740800,"data_stored":129470793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509076,"kb_used_data":58116,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93862764,"statfs":{"total":96636764160,"available":96115470336,"internally_reserved":0,"allocated":59510784,"data_stored":59175284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":128843776,"data_stored":128840163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57876480,"data_stored":57807374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181153792,"data_stored":181146107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:13.474 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:14.016 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:14.016 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:14.051 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-05T13:22:13.736207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67408398,"num_objects":16461,"num_object_clones":0,"num_object_copies":32922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41886,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274730,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11613,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599100,"kb_used_data":349004,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281516420,"statfs":{"total":289910292480,"available":288272814080,"internally_reserved":0,"allocated":357380096,"data_stored":356504587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1280035846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,6,12,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19001344,"num_objects":-4639,"num_object_clones":0,"num_object_copies":-9278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004372"},"pg_stats":[{"pgid":"2.f","version":"47'19939","reported_seq":20020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.908417+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:10.908417+0000","last_peered":"2024-09-05T13:22:10.908417+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:10.908417+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231168,"num_objects":1033,"num_object_clones":0,"num_object_copies":2066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19939,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19710","reported_seq":19821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.937831+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:08.937831+0000","last_peered":"2024-09-05T13:22:08.937831+0000","last_clean":"2024-09-05T13:22:08.937831+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:08.937831+0000","last_undegraded":"2024-09-05T13:22:08.937831+0000","last_fullsized":"2024-09-05T13:22:08.937831+0000","mapping_epoch":35,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4276224,"num_objects":1044,"num_object_clones":0,"num_object_copies":2088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17720,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20123","reported_seq":25819,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.948304+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:08.948304+0000","last_peered":"2024-09-05T13:22:08.948304+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:08.948304+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390936,"num_objects":1073,"num_object_clones":0,"num_object_copies":2146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1073,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17934,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2782,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19960","reported_seq":20036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.844265+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:08.844265+0000","last_peered":"2024-09-05T13:22:08.844265+0000","last_clean":"2024-09-05T13:22:08.844265+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:08.844265+0000","last_undegraded":"2024-09-05T13:22:08.844265+0000","last_fullsized":"2024-09-05T13:22:08.844265+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4366336,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19888","reported_seq":20029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:10.846233+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:10.846233+0000","last_peered":"2024-09-05T13:22:10.846233+0000","last_clean":"2024-09-05T13:22:10.846233+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:10.846233+0000","last_undegraded":"2024-09-05T13:22:10.846233+0000","last_fullsized":"2024-09-05T13:22:10.846233+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4075520,"num_objects":995,"num_object_clones":0,"num_object_copies":1990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16428,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20137","reported_seq":20282,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.869483+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:10.869483+0000","last_peered":"2024-09-05T13:22:10.869483+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:10.869483+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4169728,"num_objects":1018,"num_object_clones":0,"num_object_copies":2036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16592,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19635","reported_seq":28668,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:10.226504+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:10.226504+0000","last_peered":"2024-09-05T13:22:10.226504+0000","last_clean":"2024-09-05T13:22:10.226504+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:10.226504+0000","last_undegraded":"2024-09-05T13:22:10.226504+0000","last_fullsized":"2024-09-05T13:22:10.226504+0000","mapping_epoch":46,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067328,"num_objects":993,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15954,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20118","reported_seq":20256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:10.840955+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:10.840955+0000","last_peered":"2024-09-05T13:22:10.840955+0000","last_clean":"2024-09-05T13:22:10.840955+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:10.840955+0000","last_undegraded":"2024-09-05T13:22:10.840955+0000","last_fullsized":"2024-09-05T13:22:10.840955+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4038656,"num_objects":986,"num_object_clones":0,"num_object_copies":1972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16728,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19972","reported_seq":20113,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.891694+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:08.891694+0000","last_peered":"2024-09-05T13:22:08.891694+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:08.891694+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4591616,"num_objects":1121,"num_object_clones":0,"num_object_copies":2242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17626,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3330}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19925","reported_seq":20068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.776532+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:10.776532+0000","last_peered":"2024-09-05T13:22:10.776532+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:10.776532+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16292,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3469}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19870","reported_seq":28777,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.930129+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:08.930129+0000","last_peered":"2024-09-05T13:22:08.930129+0000","last_clean":"2024-09-05T13:22:08.930129+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:08.930129+0000","last_undegraded":"2024-09-05T13:22:08.930129+0000","last_fullsized":"2024-09-05T13:22:08.930129+0000","mapping_epoch":44,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4263936,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17465,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19526","reported_seq":19650,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.844798+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:08.844798+0000","last_peered":"2024-09-05T13:22:08.844798+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:08.844798+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":1056,"num_object_clones":0,"num_object_copies":2112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17035,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19872","reported_seq":20019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.818233+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:10.818233+0000","last_peered":"2024-09-05T13:22:10.818233+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:10.818233+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4120576,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16023,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20060","reported_seq":20254,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.853572+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:10.853572+0000","last_peered":"2024-09-05T13:22:10.853572+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:10.853572+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4276224,"num_objects":1044,"num_object_clones":0,"num_object_copies":2088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15959,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20078","reported_seq":20223,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.888554+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:10.888554+0000","last_peered":"2024-09-05T13:22:10.888554+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:10.888554+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4104192,"num_objects":1002,"num_object_clones":0,"num_object_copies":2004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16208,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20197","reported_seq":20343,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.894409+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:10.894409+0000","last_peered":"2024-09-05T13:22:10.894409+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:10.894409+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4076003,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16865,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":67408379,"num_objects":16459,"num_object_clones":0,"num_object_copies":32918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41886,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274728,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11612,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":353591296,"data_stored":353510892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520596,"kb_used_data":177924,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93851244,"statfs":{"total":96636764160,"available":96103673856,"internally_reserved":0,"allocated":182194176,"data_stored":181924174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584852,"kb_used_data":120068,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93786988,"statfs":{"total":96636764160,"available":96037875712,"internally_reserved":0,"allocated":122949632,"data_stored":122679625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,3,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493652,"kb_used_data":51012,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93878188,"statfs":{"total":96636764160,"available":96131264512,"internally_reserved":0,"allocated":52236288,"data_stored":51900788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121839616,"data_stored":121836003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50597888,"data_stored":50528782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181153792,"data_stored":181146107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:14.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:14.594 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:14.595 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:14.617 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-05T13:22:13.736207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67408398,"num_objects":16461,"num_object_clones":0,"num_object_copies":32922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41886,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274730,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11613,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1599100,"kb_used_data":349004,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281516420,"statfs":{"total":289910292480,"available":288272814080,"internally_reserved":0,"allocated":357380096,"data_stored":356504587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1280035846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,6,12,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19001344,"num_objects":-4639,"num_object_clones":0,"num_object_copies":-9278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004372"},"pg_stats":[{"pgid":"2.f","version":"47'19939","reported_seq":20020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.908417+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:10.908417+0000","last_peered":"2024-09-05T13:22:10.908417+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:10.908417+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231168,"num_objects":1033,"num_object_clones":0,"num_object_copies":2066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19939,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19710","reported_seq":19821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.937831+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:08.937831+0000","last_peered":"2024-09-05T13:22:08.937831+0000","last_clean":"2024-09-05T13:22:08.937831+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:08.937831+0000","last_undegraded":"2024-09-05T13:22:08.937831+0000","last_fullsized":"2024-09-05T13:22:08.937831+0000","mapping_epoch":35,"log_start":"43'9700","ondisk_log_start":"43'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4276224,"num_objects":1044,"num_object_clones":0,"num_object_copies":2088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17720,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20123","reported_seq":25819,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.948304+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:08.948304+0000","last_peered":"2024-09-05T13:22:08.948304+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:08.948304+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390936,"num_objects":1073,"num_object_clones":0,"num_object_copies":2146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1073,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17934,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2782,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19960","reported_seq":20036,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.844265+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:08.844265+0000","last_peered":"2024-09-05T13:22:08.844265+0000","last_clean":"2024-09-05T13:22:08.844265+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:08.844265+0000","last_undegraded":"2024-09-05T13:22:08.844265+0000","last_fullsized":"2024-09-05T13:22:08.844265+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4366336,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19960,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19888","reported_seq":20029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:10.846233+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:10.846233+0000","last_peered":"2024-09-05T13:22:10.846233+0000","last_clean":"2024-09-05T13:22:10.846233+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:10.846233+0000","last_undegraded":"2024-09-05T13:22:10.846233+0000","last_fullsized":"2024-09-05T13:22:10.846233+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4075520,"num_objects":995,"num_object_clones":0,"num_object_copies":1990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16428,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20137","reported_seq":20282,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.869483+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:10.869483+0000","last_peered":"2024-09-05T13:22:10.869483+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:10.869483+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4169728,"num_objects":1018,"num_object_clones":0,"num_object_copies":2036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16592,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19635","reported_seq":28668,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:10.226504+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:10.226504+0000","last_peered":"2024-09-05T13:22:10.226504+0000","last_clean":"2024-09-05T13:22:10.226504+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:10.226504+0000","last_undegraded":"2024-09-05T13:22:10.226504+0000","last_fullsized":"2024-09-05T13:22:10.226504+0000","mapping_epoch":46,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4067328,"num_objects":993,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15954,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20118","reported_seq":20256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:10.840955+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:10.840955+0000","last_peered":"2024-09-05T13:22:10.840955+0000","last_clean":"2024-09-05T13:22:10.840955+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:10.840955+0000","last_undegraded":"2024-09-05T13:22:10.840955+0000","last_fullsized":"2024-09-05T13:22:10.840955+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4038656,"num_objects":986,"num_object_clones":0,"num_object_copies":1972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16728,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19972","reported_seq":20113,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.891694+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:08.891694+0000","last_peered":"2024-09-05T13:22:08.891694+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:08.891694+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4591616,"num_objects":1121,"num_object_clones":0,"num_object_copies":2242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17626,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3330}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19925","reported_seq":20068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.776532+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:10.776532+0000","last_peered":"2024-09-05T13:22:10.776532+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:10.776532+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16292,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3469}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19870","reported_seq":28777,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:08.930129+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:08.930129+0000","last_peered":"2024-09-05T13:22:08.930129+0000","last_clean":"2024-09-05T13:22:08.930129+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:08.930129+0000","last_undegraded":"2024-09-05T13:22:08.930129+0000","last_fullsized":"2024-09-05T13:22:08.930129+0000","mapping_epoch":44,"log_start":"43'9800","ondisk_log_start":"43'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4263936,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17465,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19526","reported_seq":19650,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:08.844798+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:08.844798+0000","last_peered":"2024-09-05T13:22:08.844798+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:08.844798+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9500","ondisk_log_start":"43'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":1056,"num_object_clones":0,"num_object_copies":2112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17035,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19872","reported_seq":20019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.818233+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:10.818233+0000","last_peered":"2024-09-05T13:22:10.818233+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:10.818233+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4120576,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16023,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20060","reported_seq":20254,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.853572+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:10.853572+0000","last_peered":"2024-09-05T13:22:10.853572+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:10.853572+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4276224,"num_objects":1044,"num_object_clones":0,"num_object_copies":2088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15959,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20078","reported_seq":20223,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.888554+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:10.888554+0000","last_peered":"2024-09-05T13:22:10.888554+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:10.888554+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4104192,"num_objects":1002,"num_object_clones":0,"num_object_copies":2004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16208,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20197","reported_seq":20343,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.894409+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:10.894409+0000","last_peered":"2024-09-05T13:22:10.894409+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:10.894409+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4076003,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16865,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":67408379,"num_objects":16459,"num_object_clones":0,"num_object_copies":32918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41886,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274728,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11612,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":353591296,"data_stored":353510892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520596,"kb_used_data":177924,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93851244,"statfs":{"total":96636764160,"available":96103673856,"internally_reserved":0,"allocated":182194176,"data_stored":181924174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584852,"kb_used_data":120068,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93786988,"statfs":{"total":96636764160,"available":96037875712,"internally_reserved":0,"allocated":122949632,"data_stored":122679625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,3,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493652,"kb_used_data":51012,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93878188,"statfs":{"total":96636764160,"available":96131264512,"internally_reserved":0,"allocated":52236288,"data_stored":51900788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121839616,"data_stored":121836003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50597888,"data_stored":50528782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181153792,"data_stored":181146107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'19939", 'reported_seq': 20020, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:10.908417+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:10.908417+0000', 'last_peered': '2024-09-05T13:22:10.908417+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:10.908417+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4231168, 'num_objects': 1033, 'num_object_clones': 0, 'num_object_copies': 2066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19939, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3388}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'20123", 'reported_seq': 25819, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:08.948304+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:22:08.948304+0000', 'last_peered': '2024-09-05T13:22:08.948304+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:08.948304+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4390936, 'num_objects': 1073, 'num_object_clones': 0, 'num_object_copies': 2146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1711, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1073, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17934, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2782, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'20137", 'reported_seq': 20282, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:10.869483+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:10.869483+0000', 'last_peered': '2024-09-05T13:22:10.869483+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:10.869483+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4169728, 'num_objects': 1018, 'num_object_clones': 0, 'num_object_copies': 2036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16592, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3433}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'19972", 'reported_seq': 20113, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:08.891694+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:08.891694+0000', 'last_peered': '2024-09-05T13:22:08.891694+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:08.891694+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4591616, 'num_objects': 1121, 'num_object_clones': 0, 'num_object_copies': 2242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17626, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3330}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:14.619 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'19925", 'reported_seq': 20068, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:10.776532+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:10.776532+0000', 'last_peered': '2024-09-05T13:22:10.776532+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:10.776532+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4034560, 'num_objects': 985, 'num_object_clones': 0, 'num_object_copies': 1970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16292, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3469}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'19526", 'reported_seq': 19650, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:08.844798+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:08.844798+0000', 'last_peered': '2024-09-05T13:22:08.844798+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:08.844798+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "43'9500", 'ondisk_log_start': "43'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4325376, 'num_objects': 1056, 'num_object_clones': 0, 'num_object_copies': 2112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17035, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3331}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'19872", 'reported_seq': 20019, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:10.818233+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:10.818233+0000', 'last_peered': '2024-09-05T13:22:10.818233+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:10.818233+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4120576, 'num_objects': 1006, 'num_object_clones': 0, 'num_object_copies': 2012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16023, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3413}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'20060", 'reported_seq': 20254, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:10.853572+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:10.853572+0000', 'last_peered': '2024-09-05T13:22:10.853572+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:10.853572+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4276224, 'num_objects': 1044, 'num_object_clones': 0, 'num_object_copies': 2088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15959, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3400}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'20078", 'reported_seq': 20223, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:10.888554+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:10.888554+0000', 'last_peered': '2024-09-05T13:22:10.888554+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:10.888554+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4104192, 'num_objects': 1002, 'num_object_clones': 0, 'num_object_copies': 2004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16208, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:14.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:14.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'20197", 'reported_seq': 20343, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:10.894409+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:10.894409+0000', 'last_peered': '2024-09-05T13:22:10.894409+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:10.894409+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4076003, 'num_objects': 996, 'num_object_clones': 0, 'num_object_copies': 1992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 996, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16865, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:14.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:15.307 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:15.307 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41886/32922 objects degraded (127.228%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":662,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":16461,"data_bytes":67408398,"bytes_used":1637478400,"bytes_avail":288272814080,"bytes_total":289910292480,"degraded_objects":41886,"degraded_total":32922,"degraded_ratio":1.2722799343903772,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:15.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:15.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:15.890 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:15.892 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:15.912 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-05T13:22:15.736815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":63033870,"num_objects":15393,"num_object_clones":0,"num_object_copies":30786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275798,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11734,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1617756,"kb_used_data":337132,"kb_used_omap":12,"kb_used_meta":1280563,"kb_avail":281497764,"statfs":{"total":289910292480,"available":288253710336,"internally_reserved":0,"allocated":345223168,"data_stored":344347659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1311296518},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,4,5,11,16,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23375872,"num_objects":-5707,"num_object_clones":0,"num_object_copies":-11414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-339,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5707,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":339,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004136"},"pg_stats":[{"pgid":"2.f","version":"47'19939","reported_seq":20020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.908417+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:10.908417+0000","last_peered":"2024-09-05T13:22:10.908417+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:10.908417+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231168,"num_objects":1033,"num_object_clones":0,"num_object_copies":2066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19939,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19856","reported_seq":19967,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:13.783657+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:13.783657+0000","last_peered":"2024-09-05T13:22:13.783657+0000","last_clean":"2024-09-05T13:22:13.783657+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:13.783657+0000","last_undegraded":"2024-09-05T13:22:13.783657+0000","last_fullsized":"2024-09-05T13:22:13.783657+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3678208,"num_objects":898,"num_object_clones":0,"num_object_copies":1796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17866,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20274","reported_seq":26213,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:13.915466+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:13.915466+0000","last_peered":"2024-09-05T13:22:13.915466+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:13.915466+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3772440,"num_objects":922,"num_object_clones":0,"num_object_copies":1844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":922,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18085,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2903,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-668}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20115","reported_seq":20190,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:13.934359+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:13.934359+0000","last_peered":"2024-09-05T13:22:13.934359+0000","last_clean":"2024-09-05T13:22:13.934359+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:13.934359+0000","last_undegraded":"2024-09-05T13:22:13.934359+0000","last_fullsized":"2024-09-05T13:22:13.934359+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731456,"num_objects":911,"num_object_clones":0,"num_object_copies":1822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20115,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19888","reported_seq":20029,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:10.846233+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:10.846233+0000","last_peered":"2024-09-05T13:22:10.846233+0000","last_clean":"2024-09-05T13:22:10.846233+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:10.846233+0000","last_undegraded":"2024-09-05T13:22:10.846233+0000","last_fullsized":"2024-09-05T13:22:10.846233+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4075520,"num_objects":995,"num_object_clones":0,"num_object_copies":1990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16428,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20137","reported_seq":20282,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.869483+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:10.869483+0000","last_peered":"2024-09-05T13:22:10.869483+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:10.869483+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4169728,"num_objects":1018,"num_object_clones":0,"num_object_copies":2036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16592,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3433}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19780","reported_seq":28813,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.397214+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:15.397214+0000","last_peered":"2024-09-05T13:22:15.397214+0000","last_clean":"2024-09-05T13:22:15.397214+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:15.397214+0000","last_undegraded":"2024-09-05T13:22:15.397214+0000","last_fullsized":"2024-09-05T13:22:15.397214+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16099,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20118","reported_seq":20256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:10.840955+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:10.840955+0000","last_peered":"2024-09-05T13:22:10.840955+0000","last_clean":"2024-09-05T13:22:10.840955+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:10.840955+0000","last_undegraded":"2024-09-05T13:22:10.840955+0000","last_fullsized":"2024-09-05T13:22:10.840955+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4038656,"num_objects":986,"num_object_clones":0,"num_object_copies":1972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16728,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20116","reported_seq":20257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:13.936381+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:13.936381+0000","last_peered":"2024-09-05T13:22:13.936381+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:13.936381+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4001792,"num_objects":977,"num_object_clones":0,"num_object_copies":1954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17770,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3474}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19925","reported_seq":20068,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.776532+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:10.776532+0000","last_peered":"2024-09-05T13:22:10.776532+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:10.776532+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4034560,"num_objects":985,"num_object_clones":0,"num_object_copies":1970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16292,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3469}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20048","reported_seq":28955,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:13.904168+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:13.904168+0000","last_peered":"2024-09-05T13:22:13.904168+0000","last_clean":"2024-09-05T13:22:13.904168+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:13.904168+0000","last_undegraded":"2024-09-05T13:22:13.904168+0000","last_fullsized":"2024-09-05T13:22:13.904168+0000","mapping_epoch":44,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3534848,"num_objects":863,"num_object_clones":0,"num_object_copies":1726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17643,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19675","reported_seq":19804,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:13.918350+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:13.918350+0000","last_peered":"2024-09-05T13:22:13.918350+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:13.918350+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715072,"num_objects":907,"num_object_clones":0,"num_object_copies":1814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17184,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19872","reported_seq":20019,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.818233+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:10.818233+0000","last_peered":"2024-09-05T13:22:10.818233+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:10.818233+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4120576,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16023,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3413}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20060","reported_seq":20254,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.853572+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:10.853572+0000","last_peered":"2024-09-05T13:22:10.853572+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:10.853572+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4276224,"num_objects":1044,"num_object_clones":0,"num_object_copies":2088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15959,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20078","reported_seq":20223,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.888554+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:10.888554+0000","last_peered":"2024-09-05T13:22:10.888554+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:10.888554+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4104192,"num_objects":1002,"num_object_clones":0,"num_object_copies":2004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16208,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20197","reported_seq":20343,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:10.894409+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:10.894409+0000","last_peered":"2024-09-05T13:22:10.894409+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:10.894409+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4076003,"num_objects":996,"num_object_clones":0,"num_object_copies":1992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16865,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":63033851,"num_objects":15391,"num_object_clones":0,"num_object_copies":30782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275796,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11733,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":341381120,"data_stored":341300716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520136,"kb_used_data":173304,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93851704,"statfs":{"total":96636764160,"available":96104144896,"internally_reserved":0,"allocated":177463296,"data_stored":177193294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584852,"kb_used_data":120068,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93786988,"statfs":{"total":96636764160,"available":96037875712,"internally_reserved":0,"allocated":122949632,"data_stored":122679625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":475917994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,3,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512768,"kb_used_data":43760,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93859072,"statfs":{"total":96636764160,"available":96111689728,"internally_reserved":0,"allocated":44810240,"data_stored":44474740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121839616,"data_stored":121836003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43245568,"data_stored":43176462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":176295936,"data_stored":176288251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:18.915 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:19.471 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:19.477 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:19.498 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-05T13:22:17.737346+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":57750030,"num_objects":14103,"num_object_clones":0,"num_object_copies":28206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277088,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11734,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1619160,"kb_used_data":330216,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281496360,"statfs":{"total":289910292480,"available":288252272640,"internally_reserved":0,"allocated":338141184,"data_stored":337265675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1319816198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,5,3,13,11,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24363008,"num_objects":-5948,"num_object_clones":0,"num_object_copies":-11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-231,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5948,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003956"},"pg_stats":[{"pgid":"2.f","version":"47'20082","reported_seq":20163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.864613+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:15.864613+0000","last_peered":"2024-09-05T13:22:15.864613+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:15.864613+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3645440,"num_objects":890,"num_object_clones":0,"num_object_copies":1780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20082,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3531}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19856","reported_seq":19967,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:13.783657+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:13.783657+0000","last_peered":"2024-09-05T13:22:13.783657+0000","last_clean":"2024-09-05T13:22:13.783657+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:13.783657+0000","last_undegraded":"2024-09-05T13:22:13.783657+0000","last_fullsized":"2024-09-05T13:22:13.783657+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3678208,"num_objects":898,"num_object_clones":0,"num_object_copies":1796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17866,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20274","reported_seq":26213,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:13.915466+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:13.915466+0000","last_peered":"2024-09-05T13:22:13.915466+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:13.915466+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3772440,"num_objects":922,"num_object_clones":0,"num_object_copies":1844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":922,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18085,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2903,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-668}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20115","reported_seq":20190,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:13.934359+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:13.934359+0000","last_peered":"2024-09-05T13:22:13.934359+0000","last_clean":"2024-09-05T13:22:13.934359+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:13.934359+0000","last_undegraded":"2024-09-05T13:22:13.934359+0000","last_fullsized":"2024-09-05T13:22:13.934359+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731456,"num_objects":911,"num_object_clones":0,"num_object_copies":1822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20115,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20032","reported_seq":20173,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.937844+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:15.937844+0000","last_peered":"2024-09-05T13:22:15.937844+0000","last_clean":"2024-09-05T13:22:15.937844+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:15.937844+0000","last_undegraded":"2024-09-05T13:22:15.937844+0000","last_fullsized":"2024-09-05T13:22:15.937844+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3485696,"num_objects":851,"num_object_clones":0,"num_object_copies":1702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16572,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20282","reported_seq":20427,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.862139+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:15.862139+0000","last_peered":"2024-09-05T13:22:15.862139+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:15.862139+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3575808,"num_objects":873,"num_object_clones":0,"num_object_copies":1746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19780","reported_seq":28813,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.397214+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:15.397214+0000","last_peered":"2024-09-05T13:22:15.397214+0000","last_clean":"2024-09-05T13:22:15.397214+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:15.397214+0000","last_undegraded":"2024-09-05T13:22:15.397214+0000","last_fullsized":"2024-09-05T13:22:15.397214+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16099,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20264","reported_seq":20402,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.923413+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:15.923413+0000","last_peered":"2024-09-05T13:22:15.923413+0000","last_clean":"2024-09-05T13:22:15.923413+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:15.923413+0000","last_undegraded":"2024-09-05T13:22:15.923413+0000","last_fullsized":"2024-09-05T13:22:15.923413+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3440640,"num_objects":840,"num_object_clones":0,"num_object_copies":1680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16874,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20116","reported_seq":20257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:13.936381+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:13.936381+0000","last_peered":"2024-09-05T13:22:13.936381+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:13.936381+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4001792,"num_objects":977,"num_object_clones":0,"num_object_copies":1954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17770,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3474}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20084","reported_seq":20228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.846835+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:15.846835+0000","last_peered":"2024-09-05T13:22:15.846835+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:15.846835+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3383296,"num_objects":826,"num_object_clones":0,"num_object_copies":1652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16451,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3628}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20048","reported_seq":28955,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:13.904168+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:13.904168+0000","last_peered":"2024-09-05T13:22:13.904168+0000","last_clean":"2024-09-05T13:22:13.904168+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:13.904168+0000","last_undegraded":"2024-09-05T13:22:13.904168+0000","last_fullsized":"2024-09-05T13:22:13.904168+0000","mapping_epoch":44,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3534848,"num_objects":863,"num_object_clones":0,"num_object_copies":1726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17643,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19675","reported_seq":19804,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:13.918350+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:13.918350+0000","last_peered":"2024-09-05T13:22:13.918350+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:13.918350+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"43'9600","ondisk_log_start":"43'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3715072,"num_objects":907,"num_object_clones":0,"num_object_copies":1814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17184,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20009","reported_seq":20154,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.897451+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:15.897451+0000","last_peered":"2024-09-05T13:22:15.897451+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:15.897451+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3559424,"num_objects":869,"num_object_clones":0,"num_object_copies":1738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20213","reported_seq":20405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.916612+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:15.916612+0000","last_peered":"2024-09-05T13:22:15.916612+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:15.916612+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3649536,"num_objects":891,"num_object_clones":0,"num_object_copies":1782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16112,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3553}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20202","reported_seq":20347,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.888849+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:15.888849+0000","last_peered":"2024-09-05T13:22:15.888849+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:15.888849+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3596288,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16332,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3688}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20336","reported_seq":20482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.911192+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:15.911192+0000","last_peered":"2024-09-05T13:22:15.911192+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:15.911192+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3506659,"num_objects":857,"num_object_clones":0,"num_object_copies":1714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17004,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":57750011,"num_objects":14101,"num_object_clones":0,"num_object_copies":28202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14101,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277086,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11733,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":334323712,"data_stored":334243308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520136,"kb_used_data":173304,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93851704,"statfs":{"total":96636764160,"available":96104144896,"internally_reserved":0,"allocated":177463296,"data_stored":177193294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586256,"kb_used_data":113152,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93785584,"statfs":{"total":96636764160,"available":96036438016,"internally_reserved":0,"allocated":115867648,"data_stored":115597641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,5,2,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512768,"kb_used_data":43760,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93859072,"statfs":{"total":96636764160,"available":96111689728,"internally_reserved":0,"allocated":44810240,"data_stored":44474740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114782208,"data_stored":114778595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43245568,"data_stored":43176462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":176295936,"data_stored":176288251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:19.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:20.075 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:20.075 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:20.109 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-05T13:22:19.738063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":54088206,"num_objects":13209,"num_object_clones":0,"num_object_copies":26418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277982,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11839,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1601948,"kb_used_data":325484,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281513572,"statfs":{"total":289910292480,"available":288269897728,"internally_reserved":0,"allocated":333295616,"data_stored":332420107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1307036678},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,4,8,8,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22863872,"num_objects":-5582,"num_object_clones":0,"num_object_copies":-11164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5582,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003931"},"pg_stats":[{"pgid":"2.f","version":"47'20082","reported_seq":20163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.864613+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:15.864613+0000","last_peered":"2024-09-05T13:22:15.864613+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:15.864613+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3645440,"num_objects":890,"num_object_clones":0,"num_object_copies":1780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20082,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3531}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20003","reported_seq":20114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.948640+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:18.948640+0000","last_peered":"2024-09-05T13:22:18.948640+0000","last_clean":"2024-09-05T13:22:18.948640+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:18.948640+0000","last_undegraded":"2024-09-05T13:22:18.948640+0000","last_fullsized":"2024-09-05T13:22:18.948640+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3076096,"num_objects":751,"num_object_clones":0,"num_object_copies":1502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18013,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20427","reported_seq":26579,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.917040+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:18.917040+0000","last_peered":"2024-09-05T13:22:18.917040+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:18.917040+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145752,"num_objects":769,"num_object_clones":0,"num_object_copies":1538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18238,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3008,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20246","reported_seq":20322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.944230+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:18.944230+0000","last_peered":"2024-09-05T13:22:18.944230+0000","last_clean":"2024-09-05T13:22:18.944230+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:18.944230+0000","last_undegraded":"2024-09-05T13:22:18.944230+0000","last_fullsized":"2024-09-05T13:22:18.944230+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20246,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20032","reported_seq":20173,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.937844+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:15.937844+0000","last_peered":"2024-09-05T13:22:15.937844+0000","last_clean":"2024-09-05T13:22:15.937844+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:15.937844+0000","last_undegraded":"2024-09-05T13:22:15.937844+0000","last_fullsized":"2024-09-05T13:22:15.937844+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3485696,"num_objects":851,"num_object_clones":0,"num_object_copies":1702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16572,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20282","reported_seq":20427,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.862139+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:15.862139+0000","last_peered":"2024-09-05T13:22:15.862139+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:15.862139+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3575808,"num_objects":873,"num_object_clones":0,"num_object_copies":1746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19780","reported_seq":28813,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.397214+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:15.397214+0000","last_peered":"2024-09-05T13:22:15.397214+0000","last_clean":"2024-09-05T13:22:15.397214+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:15.397214+0000","last_undegraded":"2024-09-05T13:22:15.397214+0000","last_fullsized":"2024-09-05T13:22:15.397214+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16099,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20264","reported_seq":20402,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.923413+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:15.923413+0000","last_peered":"2024-09-05T13:22:15.923413+0000","last_clean":"2024-09-05T13:22:15.923413+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:15.923413+0000","last_undegraded":"2024-09-05T13:22:15.923413+0000","last_fullsized":"2024-09-05T13:22:15.923413+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3440640,"num_objects":840,"num_object_clones":0,"num_object_copies":1680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16874,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20277","reported_seq":20418,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.879043+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:18.879043+0000","last_peered":"2024-09-05T13:22:18.879043+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:18.879043+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":816,"num_object_clones":0,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17931,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3635}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20084","reported_seq":20228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.846835+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:15.846835+0000","last_peered":"2024-09-05T13:22:15.846835+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:15.846835+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3383296,"num_objects":826,"num_object_clones":0,"num_object_copies":1652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16451,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3628}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20196","reported_seq":29103,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.879162+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:18.879162+0000","last_peered":"2024-09-05T13:22:18.879162+0000","last_clean":"2024-09-05T13:22:18.879162+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:18.879162+0000","last_undegraded":"2024-09-05T13:22:18.879162+0000","last_fullsized":"2024-09-05T13:22:18.879162+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2928640,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17791,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19829","reported_seq":19958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.954844+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:18.954844+0000","last_peered":"2024-09-05T13:22:18.954844+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:18.954844+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3084288,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17338,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3634}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20009","reported_seq":20154,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.897451+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:15.897451+0000","last_peered":"2024-09-05T13:22:15.897451+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:15.897451+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3559424,"num_objects":869,"num_object_clones":0,"num_object_copies":1738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20213","reported_seq":20405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.916612+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:15.916612+0000","last_peered":"2024-09-05T13:22:15.916612+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:15.916612+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3649536,"num_objects":891,"num_object_clones":0,"num_object_copies":1782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16112,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3553}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20202","reported_seq":20347,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.888849+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:15.888849+0000","last_peered":"2024-09-05T13:22:15.888849+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:15.888849+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3596288,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16332,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3688}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20336","reported_seq":20482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.911192+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:15.911192+0000","last_peered":"2024-09-05T13:22:15.911192+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:15.911192+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3506659,"num_objects":857,"num_object_clones":0,"num_object_copies":1714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17004,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":54088187,"num_objects":13207,"num_object_clones":0,"num_object_copies":26414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277980,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11838,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":329506816,"data_stored":329426412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502924,"kb_used_data":168572,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93868916,"statfs":{"total":96636764160,"available":96121769984,"internally_reserved":0,"allocated":172617728,"data_stored":172347726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586256,"kb_used_data":113152,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93785584,"statfs":{"total":96636764160,"available":96036438016,"internally_reserved":0,"allocated":115867648,"data_stored":115597641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,5,2,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512768,"kb_used_data":43760,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93859072,"statfs":{"total":96636764160,"available":96111689728,"internally_reserved":0,"allocated":44810240,"data_stored":44474740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114782208,"data_stored":114778595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43245568,"data_stored":43176462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171479040,"data_stored":171471355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:20.110 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:20.110 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20082", 'reported_seq': 20163, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:15.864613+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:15.864613+0000', 'last_peered': '2024-09-05T13:22:15.864613+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:15.864613+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3645440, 'num_objects': 890, 'num_object_clones': 0, 'num_object_copies': 1780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20082, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3531}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:20.110 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:20.110 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'20427", 'reported_seq': 26579, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:18.917040+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:22:18.917040+0000', 'last_peered': '2024-09-05T13:22:18.917040+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:18.917040+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3145752, 'num_objects': 769, 'num_object_clones': 0, 'num_object_copies': 1538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1485, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 769, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18238, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3008, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -716}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'20282", 'reported_seq': 20427, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:15.862139+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:15.862139+0000', 'last_peered': '2024-09-05T13:22:15.862139+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:15.862139+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3575808, 'num_objects': 873, 'num_object_clones': 0, 'num_object_copies': 1746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16737, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3578}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'20277", 'reported_seq': 20418, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:18.879043+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:18.879043+0000', 'last_peered': '2024-09-05T13:22:18.879043+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:18.879043+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3342336, 'num_objects': 816, 'num_object_clones': 0, 'num_object_copies': 1632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17931, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20084", 'reported_seq': 20228, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:15.846835+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:15.846835+0000', 'last_peered': '2024-09-05T13:22:15.846835+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:15.846835+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3383296, 'num_objects': 826, 'num_object_clones': 0, 'num_object_copies': 1652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16451, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3628}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'19829", 'reported_seq': 19958, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:18.954844+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:18.954844+0000', 'last_peered': '2024-09-05T13:22:18.954844+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:18.954844+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3084288, 'num_objects': 753, 'num_object_clones': 0, 'num_object_copies': 1506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17338, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3634}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:20.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20009", 'reported_seq': 20154, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:15.897451+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:15.897451+0000', 'last_peered': '2024-09-05T13:22:15.897451+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:15.897451+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3559424, 'num_objects': 869, 'num_object_clones': 0, 'num_object_copies': 1738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16160, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:20.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:20.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'20213", 'reported_seq': 20405, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:15.916612+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:15.916612+0000', 'last_peered': '2024-09-05T13:22:15.916612+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:15.916612+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3649536, 'num_objects': 891, 'num_object_clones': 0, 'num_object_copies': 1782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16112, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3553}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:20.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:20.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'20202", 'reported_seq': 20347, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:15.888849+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:15.888849+0000', 'last_peered': '2024-09-05T13:22:15.888849+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:15.888849+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3596288, 'num_objects': 878, 'num_object_clones': 0, 'num_object_copies': 1756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16332, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3688}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:20.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:20.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'20336", 'reported_seq': 20482, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:15.911192+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:15.911192+0000', 'last_peered': '2024-09-05T13:22:15.911192+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:15.911192+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3506659, 'num_objects': 857, 'num_object_clones': 0, 'num_object_copies': 1714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 857, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17004, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:20.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:20.804 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:20.804 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41660/26418 objects degraded (157.696%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":667,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":13209,"data_bytes":54088206,"bytes_used":1640394752,"bytes_avail":288269897728,"bytes_total":289910292480,"degraded_objects":41660,"degraded_total":26418,"degraded_ratio":1.576955106366871,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":465},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:20.832 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:20.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:21.414 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-05T13:22:19.738063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":54088206,"num_objects":13209,"num_object_clones":0,"num_object_copies":26418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277982,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11839,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160765,"ondisk_log_size":160765,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1601948,"kb_used_data":325484,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281513572,"statfs":{"total":289910292480,"available":288269897728,"internally_reserved":0,"allocated":333295616,"data_stored":332420107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1307036678},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,4,8,8,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22863872,"num_objects":-5582,"num_object_clones":0,"num_object_copies":-11164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5582,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003931"},"pg_stats":[{"pgid":"2.f","version":"47'20082","reported_seq":20163,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.864613+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:15.864613+0000","last_peered":"2024-09-05T13:22:15.864613+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:15.864613+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3645440,"num_objects":890,"num_object_clones":0,"num_object_copies":1780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20082,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3531}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20003","reported_seq":20114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.948640+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:18.948640+0000","last_peered":"2024-09-05T13:22:18.948640+0000","last_clean":"2024-09-05T13:22:18.948640+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:18.948640+0000","last_undegraded":"2024-09-05T13:22:18.948640+0000","last_fullsized":"2024-09-05T13:22:18.948640+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3076096,"num_objects":751,"num_object_clones":0,"num_object_copies":1502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18013,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20427","reported_seq":26579,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.917040+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:18.917040+0000","last_peered":"2024-09-05T13:22:18.917040+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:18.917040+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145752,"num_objects":769,"num_object_clones":0,"num_object_copies":1538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18238,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3008,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20246","reported_seq":20322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.944230+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:18.944230+0000","last_peered":"2024-09-05T13:22:18.944230+0000","last_clean":"2024-09-05T13:22:18.944230+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:18.944230+0000","last_undegraded":"2024-09-05T13:22:18.944230+0000","last_fullsized":"2024-09-05T13:22:18.944230+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20246,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20032","reported_seq":20173,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.937844+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:15.937844+0000","last_peered":"2024-09-05T13:22:15.937844+0000","last_clean":"2024-09-05T13:22:15.937844+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:15.937844+0000","last_undegraded":"2024-09-05T13:22:15.937844+0000","last_fullsized":"2024-09-05T13:22:15.937844+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3485696,"num_objects":851,"num_object_clones":0,"num_object_copies":1702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16572,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20282","reported_seq":20427,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.862139+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:15.862139+0000","last_peered":"2024-09-05T13:22:15.862139+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:15.862139+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3575808,"num_objects":873,"num_object_clones":0,"num_object_copies":1746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19780","reported_seq":28813,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.397214+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:15.397214+0000","last_peered":"2024-09-05T13:22:15.397214+0000","last_clean":"2024-09-05T13:22:15.397214+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:15.397214+0000","last_undegraded":"2024-09-05T13:22:15.397214+0000","last_fullsized":"2024-09-05T13:22:15.397214+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16099,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20264","reported_seq":20402,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:15.923413+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:15.923413+0000","last_peered":"2024-09-05T13:22:15.923413+0000","last_clean":"2024-09-05T13:22:15.923413+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:15.923413+0000","last_undegraded":"2024-09-05T13:22:15.923413+0000","last_fullsized":"2024-09-05T13:22:15.923413+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3440640,"num_objects":840,"num_object_clones":0,"num_object_copies":1680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16874,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20277","reported_seq":20418,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.879043+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:18.879043+0000","last_peered":"2024-09-05T13:22:18.879043+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:18.879043+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":816,"num_object_clones":0,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17931,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3635}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20084","reported_seq":20228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.846835+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:15.846835+0000","last_peered":"2024-09-05T13:22:15.846835+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:15.846835+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3383296,"num_objects":826,"num_object_clones":0,"num_object_copies":1652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16451,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3628}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20196","reported_seq":29103,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.879162+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:18.879162+0000","last_peered":"2024-09-05T13:22:18.879162+0000","last_clean":"2024-09-05T13:22:18.879162+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:18.879162+0000","last_undegraded":"2024-09-05T13:22:18.879162+0000","last_fullsized":"2024-09-05T13:22:18.879162+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2928640,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17791,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19829","reported_seq":19958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.954844+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:18.954844+0000","last_peered":"2024-09-05T13:22:18.954844+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:18.954844+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3084288,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17338,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3634}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20009","reported_seq":20154,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.897451+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:15.897451+0000","last_peered":"2024-09-05T13:22:15.897451+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:15.897451+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3559424,"num_objects":869,"num_object_clones":0,"num_object_copies":1738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20213","reported_seq":20405,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.916612+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:15.916612+0000","last_peered":"2024-09-05T13:22:15.916612+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:15.916612+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3649536,"num_objects":891,"num_object_clones":0,"num_object_copies":1782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16112,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3553}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20202","reported_seq":20347,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.888849+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:15.888849+0000","last_peered":"2024-09-05T13:22:15.888849+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:15.888849+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3596288,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16332,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3688}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20336","reported_seq":20482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:15.911192+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:15.911192+0000","last_peered":"2024-09-05T13:22:15.911192+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:15.911192+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3506659,"num_objects":857,"num_object_clones":0,"num_object_copies":1714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17004,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":54088187,"num_objects":13207,"num_object_clones":0,"num_object_copies":26414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277980,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11838,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":329506816,"data_stored":329426412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502924,"kb_used_data":168572,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93868916,"statfs":{"total":96636764160,"available":96121769984,"internally_reserved":0,"allocated":172617728,"data_stored":172347726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586256,"kb_used_data":113152,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93785584,"statfs":{"total":96636764160,"available":96036438016,"internally_reserved":0,"allocated":115867648,"data_stored":115597641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,5,2,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512768,"kb_used_data":43760,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93859072,"statfs":{"total":96636764160,"available":96111689728,"internally_reserved":0,"allocated":44810240,"data_stored":44474740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114782208,"data_stored":114778595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43245568,"data_stored":43176462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171479040,"data_stored":171471355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:24.416 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:24.988 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:25.014 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-05T13:22:23.739338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48255502,"num_objects":11785,"num_object_clones":0,"num_object_copies":23570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279406,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11839,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1582764,"kb_used_data":310460,"kb_used_omap":12,"kb_used_meta":1272243,"kb_avail":281532756,"statfs":{"total":289910292480,"available":288289542144,"internally_reserved":0,"allocated":317911040,"data_stored":317035531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1302776838},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,8,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19152896,"num_objects":-4676,"num_object_clones":0,"num_object_copies":-9352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4676,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003943"},"pg_stats":[{"pgid":"2.f","version":"47'20240","reported_seq":20321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.926189+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:20.926189+0000","last_peered":"2024-09-05T13:22:20.926189+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:20.926189+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2998272,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20240,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3689}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20003","reported_seq":20114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.948640+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:18.948640+0000","last_peered":"2024-09-05T13:22:18.948640+0000","last_clean":"2024-09-05T13:22:18.948640+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:18.948640+0000","last_undegraded":"2024-09-05T13:22:18.948640+0000","last_fullsized":"2024-09-05T13:22:18.948640+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3076096,"num_objects":751,"num_object_clones":0,"num_object_copies":1502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18013,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20427","reported_seq":26579,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.917040+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:18.917040+0000","last_peered":"2024-09-05T13:22:18.917040+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:18.917040+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145752,"num_objects":769,"num_object_clones":0,"num_object_copies":1538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18238,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3008,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20246","reported_seq":20322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.944230+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:18.944230+0000","last_peered":"2024-09-05T13:22:18.944230+0000","last_clean":"2024-09-05T13:22:18.944230+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:18.944230+0000","last_undegraded":"2024-09-05T13:22:18.944230+0000","last_fullsized":"2024-09-05T13:22:18.944230+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20246,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20147","reported_seq":20288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:21.045192+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:21.045192+0000","last_peered":"2024-09-05T13:22:21.045192+0000","last_clean":"2024-09-05T13:22:21.045192+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:21.045192+0000","last_undegraded":"2024-09-05T13:22:21.045192+0000","last_fullsized":"2024-09-05T13:22:21.045192+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":736,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16687,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20429","reported_seq":20574,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.859662+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:20.859662+0000","last_peered":"2024-09-05T13:22:20.859662+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:20.859662+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2973696,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16884,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3725}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19927","reported_seq":28960,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:20.378631+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:20.378631+0000","last_peered":"2024-09-05T13:22:20.378631+0000","last_clean":"2024-09-05T13:22:20.378631+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:20.378631+0000","last_undegraded":"2024-09-05T13:22:20.378631+0000","last_fullsized":"2024-09-05T13:22:20.378631+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2871296,"num_objects":701,"num_object_clones":0,"num_object_copies":1402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16246,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20413","reported_seq":20551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:20.894362+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:20.894362+0000","last_peered":"2024-09-05T13:22:20.894362+0000","last_clean":"2024-09-05T13:22:20.894362+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:20.894362+0000","last_undegraded":"2024-09-05T13:22:20.894362+0000","last_fullsized":"2024-09-05T13:22:20.894362+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2830336,"num_objects":691,"num_object_clones":0,"num_object_copies":1382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17023,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20277","reported_seq":20418,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.879043+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:18.879043+0000","last_peered":"2024-09-05T13:22:18.879043+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:18.879043+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":816,"num_object_clones":0,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17931,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3635}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20207","reported_seq":20351,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.915368+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:20.915368+0000","last_peered":"2024-09-05T13:22:20.915368+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:20.915368+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2879488,"num_objects":703,"num_object_clones":0,"num_object_copies":1406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16574,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20196","reported_seq":29103,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.879162+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:18.879162+0000","last_peered":"2024-09-05T13:22:18.879162+0000","last_clean":"2024-09-05T13:22:18.879162+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:18.879162+0000","last_undegraded":"2024-09-05T13:22:18.879162+0000","last_fullsized":"2024-09-05T13:22:18.879162+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2928640,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17791,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19829","reported_seq":19958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.954844+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:18.954844+0000","last_peered":"2024-09-05T13:22:18.954844+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:18.954844+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3084288,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17338,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3634}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20151","reported_seq":20298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.925525+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:20.925525+0000","last_peered":"2024-09-05T13:22:20.925525+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:20.925525+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2977792,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16302,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20365","reported_seq":20559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:21.026577+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:21.026577+0000","last_peered":"2024-09-05T13:22:21.026577+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:21.026577+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16264,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20354","reported_seq":20499,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:21.027277+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:21.027277+0000","last_peered":"2024-09-05T13:22:21.027277+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:21.027277+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2973696,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16484,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20475","reported_seq":20621,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.905456+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:20.905456+0000","last_peered":"2024-09-05T13:22:20.905456+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:20.905456+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2937315,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17143,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":48255483,"num_objects":11783,"num_object_clones":0,"num_object_copies":23566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279404,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11838,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315088896,"data_stored":315008492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502924,"kb_used_data":168572,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93868916,"statfs":{"total":96636764160,"available":96121769984,"internally_reserved":0,"allocated":172617728,"data_stored":172347726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567008,"kb_used_data":106384,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93804832,"statfs":{"total":96636764160,"available":96056147968,"internally_reserved":0,"allocated":108937216,"data_stored":108667209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512832,"kb_used_data":35504,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93859008,"statfs":{"total":96636764160,"available":96111624192,"internally_reserved":0,"allocated":36356096,"data_stored":36020596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":488763055},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107761664,"data_stored":107758051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35848192,"data_stored":35779086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171479040,"data_stored":171471355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:25.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:25.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:25.571 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:25.598 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-05T13:22:23.739338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48255502,"num_objects":11785,"num_object_clones":0,"num_object_copies":23570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279406,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11839,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1582764,"kb_used_data":310460,"kb_used_omap":12,"kb_used_meta":1272243,"kb_avail":281532756,"statfs":{"total":289910292480,"available":288289542144,"internally_reserved":0,"allocated":317911040,"data_stored":317035531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1302776838},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,8,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19152896,"num_objects":-4676,"num_object_clones":0,"num_object_copies":-9352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4676,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003943"},"pg_stats":[{"pgid":"2.f","version":"47'20240","reported_seq":20321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.926189+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:20.926189+0000","last_peered":"2024-09-05T13:22:20.926189+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:20.926189+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2998272,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20240,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3689}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20003","reported_seq":20114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.948640+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:18.948640+0000","last_peered":"2024-09-05T13:22:18.948640+0000","last_clean":"2024-09-05T13:22:18.948640+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:18.948640+0000","last_undegraded":"2024-09-05T13:22:18.948640+0000","last_fullsized":"2024-09-05T13:22:18.948640+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3076096,"num_objects":751,"num_object_clones":0,"num_object_copies":1502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18013,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20427","reported_seq":26579,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.917040+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:18.917040+0000","last_peered":"2024-09-05T13:22:18.917040+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:18.917040+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145752,"num_objects":769,"num_object_clones":0,"num_object_copies":1538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":769,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18238,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3008,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20246","reported_seq":20322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.944230+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:18.944230+0000","last_peered":"2024-09-05T13:22:18.944230+0000","last_clean":"2024-09-05T13:22:18.944230+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:18.944230+0000","last_undegraded":"2024-09-05T13:22:18.944230+0000","last_fullsized":"2024-09-05T13:22:18.944230+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3194880,"num_objects":780,"num_object_clones":0,"num_object_copies":1560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20246,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20147","reported_seq":20288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:21.045192+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:21.045192+0000","last_peered":"2024-09-05T13:22:21.045192+0000","last_clean":"2024-09-05T13:22:21.045192+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:21.045192+0000","last_undegraded":"2024-09-05T13:22:21.045192+0000","last_fullsized":"2024-09-05T13:22:21.045192+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":736,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16687,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20429","reported_seq":20574,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.859662+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:20.859662+0000","last_peered":"2024-09-05T13:22:20.859662+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:20.859662+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2973696,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16884,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3725}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19927","reported_seq":28960,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:20.378631+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:20.378631+0000","last_peered":"2024-09-05T13:22:20.378631+0000","last_clean":"2024-09-05T13:22:20.378631+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:20.378631+0000","last_undegraded":"2024-09-05T13:22:20.378631+0000","last_fullsized":"2024-09-05T13:22:20.378631+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2871296,"num_objects":701,"num_object_clones":0,"num_object_copies":1402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16246,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20413","reported_seq":20551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:20.894362+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:20.894362+0000","last_peered":"2024-09-05T13:22:20.894362+0000","last_clean":"2024-09-05T13:22:20.894362+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:20.894362+0000","last_undegraded":"2024-09-05T13:22:20.894362+0000","last_fullsized":"2024-09-05T13:22:20.894362+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2830336,"num_objects":691,"num_object_clones":0,"num_object_copies":1382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17023,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20277","reported_seq":20418,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.879043+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:18.879043+0000","last_peered":"2024-09-05T13:22:18.879043+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:18.879043+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342336,"num_objects":816,"num_object_clones":0,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17931,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3635}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20207","reported_seq":20351,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.915368+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:20.915368+0000","last_peered":"2024-09-05T13:22:20.915368+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:20.915368+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2879488,"num_objects":703,"num_object_clones":0,"num_object_copies":1406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16574,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20196","reported_seq":29103,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:18.879162+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:18.879162+0000","last_peered":"2024-09-05T13:22:18.879162+0000","last_clean":"2024-09-05T13:22:18.879162+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:18.879162+0000","last_undegraded":"2024-09-05T13:22:18.879162+0000","last_fullsized":"2024-09-05T13:22:18.879162+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2928640,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17791,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19829","reported_seq":19958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:18.954844+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:18.954844+0000","last_peered":"2024-09-05T13:22:18.954844+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:18.954844+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3084288,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17338,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3634}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20151","reported_seq":20298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.925525+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:20.925525+0000","last_peered":"2024-09-05T13:22:20.925525+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:20.925525+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2977792,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16302,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20365","reported_seq":20559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:21.026577+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:21.026577+0000","last_peered":"2024-09-05T13:22:21.026577+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:21.026577+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16264,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20354","reported_seq":20499,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:21.027277+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:21.027277+0000","last_peered":"2024-09-05T13:22:21.027277+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:21.027277+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2973696,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16484,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20475","reported_seq":20621,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.905456+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:20.905456+0000","last_peered":"2024-09-05T13:22:20.905456+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:20.905456+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2937315,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17143,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":48255483,"num_objects":11783,"num_object_clones":0,"num_object_copies":23566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41660,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279404,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11838,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315088896,"data_stored":315008492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502924,"kb_used_data":168572,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93868916,"statfs":{"total":96636764160,"available":96121769984,"internally_reserved":0,"allocated":172617728,"data_stored":172347726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567008,"kb_used_data":106384,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93804832,"statfs":{"total":96636764160,"available":96056147968,"internally_reserved":0,"allocated":108937216,"data_stored":108667209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512832,"kb_used_data":35504,"kb_used_omap":4,"kb_used_meta":477307,"kb_avail":93859008,"statfs":{"total":96636764160,"available":96111624192,"internally_reserved":0,"allocated":36356096,"data_stored":36020596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":488763055},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107761664,"data_stored":107758051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35848192,"data_stored":35779086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171479040,"data_stored":171471355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:25.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:25.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20240", 'reported_seq': 20321, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:20.926189+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:20.926189+0000', 'last_peered': '2024-09-05T13:22:20.926189+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:20.926189+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2998272, 'num_objects': 732, 'num_object_clones': 0, 'num_object_copies': 1464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20240, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3689}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:25.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:25.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'20427", 'reported_seq': 26579, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:18.917040+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:22:18.917040+0000', 'last_peered': '2024-09-05T13:22:18.917040+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:18.917040+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3145752, 'num_objects': 769, 'num_object_clones': 0, 'num_object_copies': 1538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1485, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 769, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18238, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3008, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -716}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:25.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:25.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'20429", 'reported_seq': 20574, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:20.859662+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:20.859662+0000', 'last_peered': '2024-09-05T13:22:20.859662+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:20.859662+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2973696, 'num_objects': 726, 'num_object_clones': 0, 'num_object_copies': 1452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16884, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3725}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:25.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:25.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'20277", 'reported_seq': 20418, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:18.879043+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:18.879043+0000', 'last_peered': '2024-09-05T13:22:18.879043+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:18.879043+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3342336, 'num_objects': 816, 'num_object_clones': 0, 'num_object_copies': 1632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17931, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:25.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:25.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20207", 'reported_seq': 20351, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:20.915368+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:20.915368+0000', 'last_peered': '2024-09-05T13:22:20.915368+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:20.915368+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2879488, 'num_objects': 703, 'num_object_clones': 0, 'num_object_copies': 1406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 703, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16574, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3751}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:25.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:25.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'19829", 'reported_seq': 19958, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:18.954844+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:18.954844+0000', 'last_peered': '2024-09-05T13:22:18.954844+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:18.954844+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3084288, 'num_objects': 753, 'num_object_clones': 0, 'num_object_copies': 1506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17338, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3634}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:25.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:25.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20151", 'reported_seq': 20298, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:20.925525+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:20.925525+0000', 'last_peered': '2024-09-05T13:22:20.925525+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:20.925525+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2977792, 'num_objects': 727, 'num_object_clones': 0, 'num_object_copies': 1454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16302, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3692}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:25.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:25.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'20365", 'reported_seq': 20559, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:21.026577+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:21.026577+0000', 'last_peered': '2024-09-05T13:22:21.026577+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:21.026577+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3026944, 'num_objects': 739, 'num_object_clones': 0, 'num_object_copies': 1478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16264, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3705}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:25.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:25.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'20354", 'reported_seq': 20499, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:21.027277+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:21.027277+0000', 'last_peered': '2024-09-05T13:22:21.027277+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:21.027277+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2973696, 'num_objects': 726, 'num_object_clones': 0, 'num_object_copies': 1452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16484, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:25.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:25.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'20475", 'reported_seq': 20621, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:20.905456+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:20.905456+0000', 'last_peered': '2024-09-05T13:22:20.905456+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:20.905456+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2937315, 'num_objects': 718, 'num_object_clones': 0, 'num_object_copies': 1436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17143, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3847}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:25.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:26.333 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:26.334 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41660/23570 objects degraded (176.750%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":673,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":11785,"data_bytes":48255502,"bytes_used":1620750336,"bytes_avail":288289542144,"bytes_total":289910292480,"degraded_objects":41660,"degraded_total":23570,"degraded_ratio":1.7675010606703436,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":389},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:26.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:26.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:26.937 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:26.940 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:26.965 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-05T13:22:25.739939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":44249614,"num_objects":10807,"num_object_clones":0,"num_object_copies":21614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280384,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11951,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1563104,"kb_used_data":299120,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281552416,"statfs":{"total":289910292480,"available":288309673984,"internally_reserved":0,"allocated":306298880,"data_stored":305423371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1294257158},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,6,7,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23158784,"num_objects":-5654,"num_object_clones":0,"num_object_copies":-11308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5654,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003732"},"pg_stats":[{"pgid":"2.f","version":"47'20240","reported_seq":20321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.926189+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:20.926189+0000","last_peered":"2024-09-05T13:22:20.926189+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:20.926189+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2998272,"num_objects":732,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20240,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3689}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20139","reported_seq":20250,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:23.945973+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:23.945973+0000","last_peered":"2024-09-05T13:22:23.945973+0000","last_clean":"2024-09-05T13:22:23.945973+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:23.945973+0000","last_undegraded":"2024-09-05T13:22:23.945973+0000","last_fullsized":"2024-09-05T13:22:23.945973+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519040,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18149,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20572","reported_seq":26948,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:23.970982+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:23.970982+0000","last_peered":"2024-09-05T13:22:23.970982+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:23.970982+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2551832,"num_objects":624,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18383,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-749}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20400","reported_seq":20476,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:23.969780+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:23.969780+0000","last_peered":"2024-09-05T13:22:23.969780+0000","last_clean":"2024-09-05T13:22:23.969780+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:23.969780+0000","last_undegraded":"2024-09-05T13:22:23.969780+0000","last_fullsized":"2024-09-05T13:22:23.969780+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2564096,"num_objects":626,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20400,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20147","reported_seq":20288,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:21.045192+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:21.045192+0000","last_peered":"2024-09-05T13:22:21.045192+0000","last_clean":"2024-09-05T13:22:21.045192+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:21.045192+0000","last_undegraded":"2024-09-05T13:22:21.045192+0000","last_fullsized":"2024-09-05T13:22:21.045192+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":736,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16687,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20429","reported_seq":20574,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.859662+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:20.859662+0000","last_peered":"2024-09-05T13:22:20.859662+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:20.859662+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2973696,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16884,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3725}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20056","reported_seq":29089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:25.527055+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:25.527055+0000","last_peered":"2024-09-05T13:22:25.527055+0000","last_clean":"2024-09-05T13:22:25.527055+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:25.527055+0000","last_undegraded":"2024-09-05T13:22:25.527055+0000","last_fullsized":"2024-09-05T13:22:25.527055+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342912,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16375,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20413","reported_seq":20551,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:20.894362+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:20.894362+0000","last_peered":"2024-09-05T13:22:20.894362+0000","last_clean":"2024-09-05T13:22:20.894362+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:20.894362+0000","last_undegraded":"2024-09-05T13:22:20.894362+0000","last_fullsized":"2024-09-05T13:22:20.894362+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2830336,"num_objects":691,"num_object_clones":0,"num_object_copies":1382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17023,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20422","reported_seq":20563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:23.896780+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:23.896780+0000","last_peered":"2024-09-05T13:22:23.896780+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:23.896780+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2748416,"num_objects":671,"num_object_clones":0,"num_object_copies":1342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18076,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3780}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20207","reported_seq":20351,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.915368+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:20.915368+0000","last_peered":"2024-09-05T13:22:20.915368+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:20.915368+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2879488,"num_objects":703,"num_object_clones":0,"num_object_copies":1406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16574,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20327","reported_seq":29234,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:23.935123+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:23.935123+0000","last_peered":"2024-09-05T13:22:23.935123+0000","last_clean":"2024-09-05T13:22:23.935123+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:23.935123+0000","last_undegraded":"2024-09-05T13:22:23.935123+0000","last_fullsized":"2024-09-05T13:22:23.935123+0000","mapping_epoch":44,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2392064,"num_objects":584,"num_object_clones":0,"num_object_copies":1168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17922,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19967","reported_seq":20097,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:23.954027+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:23.954027+0000","last_peered":"2024-09-05T13:22:23.954027+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:23.954027+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519040,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17476,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3772}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20151","reported_seq":20298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.925525+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:20.925525+0000","last_peered":"2024-09-05T13:22:20.925525+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:20.925525+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2977792,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16302,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20365","reported_seq":20559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:21.026577+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:21.026577+0000","last_peered":"2024-09-05T13:22:21.026577+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:21.026577+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16264,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20354","reported_seq":20499,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:21.027277+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:21.027277+0000","last_peered":"2024-09-05T13:22:21.027277+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:21.027277+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2973696,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16484,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20475","reported_seq":20621,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:20.905456+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:20.905456+0000","last_peered":"2024-09-05T13:22:20.905456+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:20.905456+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2937315,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17143,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":44249595,"num_objects":10805,"num_object_clones":0,"num_object_copies":21610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280382,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11950,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303263744,"data_stored":303183340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160764,"ondisk_log_size":160764,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502600,"kb_used_data":164088,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93869240,"statfs":{"total":96636764160,"available":96122101760,"internally_reserved":0,"allocated":168026112,"data_stored":167756110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567008,"kb_used_data":106384,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93804832,"statfs":{"total":96636764160,"available":96056147968,"internally_reserved":0,"allocated":108937216,"data_stored":108667209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":471658154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493496,"kb_used_data":28648,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93878344,"statfs":{"total":96636764160,"available":96131424256,"internally_reserved":0,"allocated":29335552,"data_stored":29000052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107761664,"data_stored":107758051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28643854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166781435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:29.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:30.529 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:30.529 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:30.554 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-05T13:22:29.742908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35058190,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282628,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12063,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1563968,"kb_used_data":287500,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281551552,"statfs":{"total":289910292480,"available":288308789248,"internally_reserved":0,"allocated":294400000,"data_stored":293524491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1307036678},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,4,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22691840,"num_objects":-5540,"num_object_clones":0,"num_object_copies":-11080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005562"},"pg_stats":[{"pgid":"2.f","version":"47'20390","reported_seq":20471,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:25.924094+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:25.924094+0000","last_peered":"2024-09-05T13:22:25.924094+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:25.924094+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2383872,"num_objects":582,"num_object_clones":0,"num_object_copies":1164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20390,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3839}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20274","reported_seq":20385,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.995262+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:28.995262+0000","last_peered":"2024-09-05T13:22:28.995262+0000","last_clean":"2024-09-05T13:22:28.995262+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:28.995262+0000","last_undegraded":"2024-09-05T13:22:28.995262+0000","last_fullsized":"2024-09-05T13:22:28.995262+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":480,"num_object_clones":0,"num_object_copies":960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18284,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20722","reported_seq":27320,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:29.007809+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:29.007809+0000","last_peered":"2024-09-05T13:22:29.007809+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:29.007809+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1937432,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18533,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20532","reported_seq":20608,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.986120+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:28.986120+0000","last_peered":"2024-09-05T13:22:28.986120+0000","last_clean":"2024-09-05T13:22:28.986120+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:28.986120+0000","last_undegraded":"2024-09-05T13:22:28.986120+0000","last_fullsized":"2024-09-05T13:22:28.986120+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2023424,"num_objects":494,"num_object_clones":0,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20532,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20302","reported_seq":20443,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:26.020737+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:26.020737+0000","last_peered":"2024-09-05T13:22:26.020737+0000","last_clean":"2024-09-05T13:22:26.020737+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:26.020737+0000","last_undegraded":"2024-09-05T13:22:26.020737+0000","last_fullsized":"2024-09-05T13:22:26.020737+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379776,"num_objects":581,"num_object_clones":0,"num_object_copies":1162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16842,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20603","reported_seq":20748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.020853+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:26.020853+0000","last_peered":"2024-09-05T13:22:26.020853+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:26.020853+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260992,"num_objects":552,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17058,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3899}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20056","reported_seq":29089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:25.527055+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:25.527055+0000","last_peered":"2024-09-05T13:22:25.527055+0000","last_clean":"2024-09-05T13:22:25.527055+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:25.527055+0000","last_undegraded":"2024-09-05T13:22:25.527055+0000","last_fullsized":"2024-09-05T13:22:25.527055+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342912,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16375,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20579","reported_seq":20717,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:25.998167+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:25.998167+0000","last_peered":"2024-09-05T13:22:25.998167+0000","last_clean":"2024-09-05T13:22:25.998167+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:25.998167+0000","last_undegraded":"2024-09-05T13:22:25.998167+0000","last_fullsized":"2024-09-05T13:22:25.998167+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2150400,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17189,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20580","reported_seq":20721,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:28.952272+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:28.952272+0000","last_peered":"2024-09-05T13:22:28.952272+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:28.952272+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2101248,"num_objects":513,"num_object_clones":0,"num_object_copies":1026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18234,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20338","reported_seq":20482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.021661+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:26.021661+0000","last_peered":"2024-09-05T13:22:26.021661+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:26.021661+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342912,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16705,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3882}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20474","reported_seq":29381,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.940889+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:28.940889+0000","last_peered":"2024-09-05T13:22:28.940889+0000","last_clean":"2024-09-05T13:22:28.940889+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:28.940889+0000","last_undegraded":"2024-09-05T13:22:28.940889+0000","last_fullsized":"2024-09-05T13:22:28.940889+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1789952,"num_objects":437,"num_object_clones":0,"num_object_copies":874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18069,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20115","reported_seq":20245,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:28.946724+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:28.946724+0000","last_peered":"2024-09-05T13:22:28.946724+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:28.946724+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1912832,"num_objects":467,"num_object_clones":0,"num_object_copies":934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17624,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3920}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20303","reported_seq":20450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.063118+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:26.063118+0000","last_peered":"2024-09-05T13:22:26.063118+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:26.063118+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2355200,"num_objects":575,"num_object_clones":0,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16454,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20512","reported_seq":20706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.065649+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:26.065649+0000","last_peered":"2024-09-05T13:22:26.065649+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:26.065649+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20499","reported_seq":20644,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:25.923120+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:25.923120+0000","last_peered":"2024-09-05T13:22:25.923120+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:25.923120+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379776,"num_objects":581,"num_object_clones":0,"num_object_copies":1162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16629,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20629","reported_seq":20774,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.033563+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:26.033563+0000","last_peered":"2024-09-05T13:22:26.033563+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:26.033563+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2306531,"num_objects":564,"num_object_clones":0,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17297,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":35058171,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282626,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12062,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":291340288,"data_stored":291259884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502256,"kb_used_data":159580,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93869584,"statfs":{"total":96636764160,"available":96122454016,"internally_reserved":0,"allocated":163409920,"data_stored":163139918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568216,"kb_used_data":99272,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93803624,"statfs":{"total":96636764160,"available":96054910976,"internally_reserved":0,"allocated":101654528,"data_stored":101384521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":480177834},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493496,"kb_used_data":28648,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93878344,"statfs":{"total":96636764160,"available":96131424256,"internally_reserved":0,"allocated":29335552,"data_stored":29000052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100511744,"data_stored":100508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28643854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162115584,"data_stored":162107899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:30.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:31.126 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:31.147 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-05T13:22:29.742908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35058190,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282628,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12063,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1563968,"kb_used_data":287500,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281551552,"statfs":{"total":289910292480,"available":288308789248,"internally_reserved":0,"allocated":294400000,"data_stored":293524491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1307036678},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,4,6,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22691840,"num_objects":-5540,"num_object_clones":0,"num_object_copies":-11080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005562"},"pg_stats":[{"pgid":"2.f","version":"47'20390","reported_seq":20471,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:25.924094+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:25.924094+0000","last_peered":"2024-09-05T13:22:25.924094+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:25.924094+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2383872,"num_objects":582,"num_object_clones":0,"num_object_copies":1164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20390,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3839}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20274","reported_seq":20385,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.995262+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:28.995262+0000","last_peered":"2024-09-05T13:22:28.995262+0000","last_clean":"2024-09-05T13:22:28.995262+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:28.995262+0000","last_undegraded":"2024-09-05T13:22:28.995262+0000","last_fullsized":"2024-09-05T13:22:28.995262+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":480,"num_object_clones":0,"num_object_copies":960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18284,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20722","reported_seq":27320,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:29.007809+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:29.007809+0000","last_peered":"2024-09-05T13:22:29.007809+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:29.007809+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1937432,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18533,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20532","reported_seq":20608,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.986120+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:28.986120+0000","last_peered":"2024-09-05T13:22:28.986120+0000","last_clean":"2024-09-05T13:22:28.986120+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:28.986120+0000","last_undegraded":"2024-09-05T13:22:28.986120+0000","last_fullsized":"2024-09-05T13:22:28.986120+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2023424,"num_objects":494,"num_object_clones":0,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20532,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20302","reported_seq":20443,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:26.020737+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:26.020737+0000","last_peered":"2024-09-05T13:22:26.020737+0000","last_clean":"2024-09-05T13:22:26.020737+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:26.020737+0000","last_undegraded":"2024-09-05T13:22:26.020737+0000","last_fullsized":"2024-09-05T13:22:26.020737+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379776,"num_objects":581,"num_object_clones":0,"num_object_copies":1162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16842,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20603","reported_seq":20748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.020853+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:26.020853+0000","last_peered":"2024-09-05T13:22:26.020853+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:26.020853+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260992,"num_objects":552,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17058,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3899}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20056","reported_seq":29089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:25.527055+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:25.527055+0000","last_peered":"2024-09-05T13:22:25.527055+0000","last_clean":"2024-09-05T13:22:25.527055+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:25.527055+0000","last_undegraded":"2024-09-05T13:22:25.527055+0000","last_fullsized":"2024-09-05T13:22:25.527055+0000","mapping_epoch":46,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342912,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16375,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20579","reported_seq":20717,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:25.998167+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:25.998167+0000","last_peered":"2024-09-05T13:22:25.998167+0000","last_clean":"2024-09-05T13:22:25.998167+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:25.998167+0000","last_undegraded":"2024-09-05T13:22:25.998167+0000","last_fullsized":"2024-09-05T13:22:25.998167+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2150400,"num_objects":525,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17189,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20580","reported_seq":20721,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:28.952272+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:28.952272+0000","last_peered":"2024-09-05T13:22:28.952272+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:28.952272+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2101248,"num_objects":513,"num_object_clones":0,"num_object_copies":1026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18234,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20338","reported_seq":20482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.021661+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:26.021661+0000","last_peered":"2024-09-05T13:22:26.021661+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:26.021661+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2342912,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16705,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3882}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106804+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:19:36.106804+0000","last_peered":"2024-09-05T13:19:36.106804+0000","last_clean":"2024-09-05T13:19:36.106804+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:19:36.106804+0000","last_undegraded":"2024-09-05T13:19:36.106804+0000","last_fullsized":"2024-09-05T13:19:36.106804+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20474","reported_seq":29381,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.940889+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:28.940889+0000","last_peered":"2024-09-05T13:22:28.940889+0000","last_clean":"2024-09-05T13:22:28.940889+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:28.940889+0000","last_undegraded":"2024-09-05T13:22:28.940889+0000","last_fullsized":"2024-09-05T13:22:28.940889+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1789952,"num_objects":437,"num_object_clones":0,"num_object_copies":874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18069,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20115","reported_seq":20245,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:28.946724+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:28.946724+0000","last_peered":"2024-09-05T13:22:28.946724+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:28.946724+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1912832,"num_objects":467,"num_object_clones":0,"num_object_copies":934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17624,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3920}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20303","reported_seq":20450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.063118+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:26.063118+0000","last_peered":"2024-09-05T13:22:26.063118+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:26.063118+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2355200,"num_objects":575,"num_object_clones":0,"num_object_copies":1150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16454,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20512","reported_seq":20706,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.065649+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:26.065649+0000","last_peered":"2024-09-05T13:22:26.065649+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:26.065649+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20499","reported_seq":20644,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:25.923120+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:25.923120+0000","last_peered":"2024-09-05T13:22:25.923120+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:25.923120+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379776,"num_objects":581,"num_object_clones":0,"num_object_copies":1162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16629,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20629","reported_seq":20774,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:26.033563+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:26.033563+0000","last_peered":"2024-09-05T13:22:26.033563+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:26.033563+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2306531,"num_objects":564,"num_object_clones":0,"num_object_copies":1128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17297,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":35058171,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282626,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12062,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":291340288,"data_stored":291259884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160708,"ondisk_log_size":160708,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502256,"kb_used_data":159580,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93869584,"statfs":{"total":96636764160,"available":96122454016,"internally_reserved":0,"allocated":163409920,"data_stored":163139918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568216,"kb_used_data":99272,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93803624,"statfs":{"total":96636764160,"available":96054910976,"internally_reserved":0,"allocated":101654528,"data_stored":101384521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":480177834},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493496,"kb_used_data":28648,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93878344,"statfs":{"total":96636764160,"available":96131424256,"internally_reserved":0,"allocated":29335552,"data_stored":29000052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100511744,"data_stored":100508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28643854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162115584,"data_stored":162107899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:31.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:31.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20390", 'reported_seq': 20471, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:25.924094+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:25.924094+0000', 'last_peered': '2024-09-05T13:22:25.924094+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:25.924094+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2383872, 'num_objects': 582, 'num_object_clones': 0, 'num_object_copies': 1164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20390, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3839}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:31.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:31.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'20722", 'reported_seq': 27320, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:29.007809+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:22:29.007809+0000', 'last_peered': '2024-09-05T13:22:29.007809+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:29.007809+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1937432, 'num_objects': 474, 'num_object_clones': 0, 'num_object_copies': 948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1261, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 474, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18533, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3232, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -787}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'20603", 'reported_seq': 20748, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:26.020853+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:26.020853+0000', 'last_peered': '2024-09-05T13:22:26.020853+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:26.020853+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2260992, 'num_objects': 552, 'num_object_clones': 0, 'num_object_copies': 1104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17058, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3899}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'20580", 'reported_seq': 20721, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:28.952272+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:28.952272+0000', 'last_peered': '2024-09-05T13:22:28.952272+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:28.952272+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2101248, 'num_objects': 513, 'num_object_clones': 0, 'num_object_copies': 1026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 513, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18234, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3938}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20338", 'reported_seq': 20482, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:26.021661+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:26.021661+0000', 'last_peered': '2024-09-05T13:22:26.021661+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:26.021661+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2342912, 'num_objects': 572, 'num_object_clones': 0, 'num_object_copies': 1144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16705, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3882}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20115", 'reported_seq': 20245, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:28.946724+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:28.946724+0000', 'last_peered': '2024-09-05T13:22:28.946724+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:28.946724+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1912832, 'num_objects': 467, 'num_object_clones': 0, 'num_object_copies': 934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17624, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3920}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:31.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:31.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20303", 'reported_seq': 20450, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:26.063118+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:26.063118+0000', 'last_peered': '2024-09-05T13:22:26.063118+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:26.063118+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2355200, 'num_objects': 575, 'num_object_clones': 0, 'num_object_copies': 1150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16454, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3844}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:31.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:31.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'20512", 'reported_seq': 20706, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:26.065649+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:26.065649+0000', 'last_peered': '2024-09-05T13:22:26.065649+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:26.065649+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2424832, 'num_objects': 592, 'num_object_clones': 0, 'num_object_copies': 1184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16411, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3852}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:31.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:31.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'20499", 'reported_seq': 20644, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:25.923120+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:25.923120+0000', 'last_peered': '2024-09-05T13:22:25.923120+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:25.923120+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2379776, 'num_objects': 581, 'num_object_clones': 0, 'num_object_copies': 1162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16629, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3985}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:31.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:31.153 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'20629", 'reported_seq': 20774, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:26.033563+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:26.033563+0000', 'last_peered': '2024-09-05T13:22:26.033563+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:26.033563+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2306531, 'num_objects': 564, 'num_object_clones': 0, 'num_object_copies': 1128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 564, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17297, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4001}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:31.153 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:31.851 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:31.851 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41436/17126 objects degraded (241.948%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":678,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":8563,"data_bytes":35058190,"bytes_used":1601503232,"bytes_avail":288308789248,"bytes_total":289910292480,"degraded_objects":41436,"degraded_total":17126,"degraded_ratio":2.4194791545019267,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":461},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:31.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:31.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:32.444 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:32.446 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:32.470 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-05T13:22:31.743483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29082126,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284087,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12063,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1562444,"kb_used_data":273496,"kb_used_omap":12,"kb_used_meta":1288883,"kb_avail":281553076,"statfs":{"total":289910292480,"available":288310349824,"internally_reserved":0,"allocated":280059904,"data_stored":279184395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1319816198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,6,10,5,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25006080,"num_objects":-6105,"num_object_clones":0,"num_object_copies":-12210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6105,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005420"},"pg_stats":[{"pgid":"2.f","version":"47'20526","reported_seq":20607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.067745+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:31.067745+0000","last_peered":"2024-09-05T13:22:31.067745+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:31.067745+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1826816,"num_objects":446,"num_object_clones":0,"num_object_copies":892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20526,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3975}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20274","reported_seq":20385,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.995262+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:28.995262+0000","last_peered":"2024-09-05T13:22:28.995262+0000","last_clean":"2024-09-05T13:22:28.995262+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:28.995262+0000","last_undegraded":"2024-09-05T13:22:28.995262+0000","last_fullsized":"2024-09-05T13:22:28.995262+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":480,"num_object_clones":0,"num_object_copies":960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18284,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20722","reported_seq":27320,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:29.007809+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:29.007809+0000","last_peered":"2024-09-05T13:22:29.007809+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:29.007809+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1937432,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18533,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20532","reported_seq":20608,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.986120+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:28.986120+0000","last_peered":"2024-09-05T13:22:28.986120+0000","last_clean":"2024-09-05T13:22:28.986120+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:28.986120+0000","last_undegraded":"2024-09-05T13:22:28.986120+0000","last_fullsized":"2024-09-05T13:22:28.986120+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2023424,"num_objects":494,"num_object_clones":0,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20532,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20447","reported_seq":20588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:31.093911+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:31.093911+0000","last_peered":"2024-09-05T13:22:31.093911+0000","last_clean":"2024-09-05T13:22:31.093911+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:31.093911+0000","last_undegraded":"2024-09-05T13:22:31.093911+0000","last_fullsized":"2024-09-05T13:22:31.093911+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1785856,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16987,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20748","reported_seq":20893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.041097+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:31.041097+0000","last_peered":"2024-09-05T13:22:31.041097+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:31.041097+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667072,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20213","reported_seq":29246,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:30.548430+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:30.548430+0000","last_peered":"2024-09-05T13:22:30.548430+0000","last_clean":"2024-09-05T13:22:30.548430+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:30.548430+0000","last_undegraded":"2024-09-05T13:22:30.548430+0000","last_fullsized":"2024-09-05T13:22:30.548430+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16532,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20706","reported_seq":20844,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:30.942131+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:30.942131+0000","last_peered":"2024-09-05T13:22:30.942131+0000","last_clean":"2024-09-05T13:22:30.942131+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:30.942131+0000","last_undegraded":"2024-09-05T13:22:30.942131+0000","last_fullsized":"2024-09-05T13:22:30.942131+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17316,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20580","reported_seq":20721,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:28.952272+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:28.952272+0000","last_peered":"2024-09-05T13:22:28.952272+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:28.952272+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2101248,"num_objects":513,"num_object_clones":0,"num_object_copies":1026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18234,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20490","reported_seq":20634,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.081671+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:31.081671+0000","last_peered":"2024-09-05T13:22:31.081671+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:31.081671+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1720320,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16857,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4034}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20474","reported_seq":29381,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:28.940889+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:28.940889+0000","last_peered":"2024-09-05T13:22:28.940889+0000","last_clean":"2024-09-05T13:22:28.940889+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:28.940889+0000","last_undegraded":"2024-09-05T13:22:28.940889+0000","last_fullsized":"2024-09-05T13:22:28.940889+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1789952,"num_objects":437,"num_object_clones":0,"num_object_copies":874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18069,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20115","reported_seq":20245,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:28.946724+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:28.946724+0000","last_peered":"2024-09-05T13:22:28.946724+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:28.946724+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1912832,"num_objects":467,"num_object_clones":0,"num_object_copies":934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17624,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3920}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20436","reported_seq":20583,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:30.924165+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:30.924165+0000","last_peered":"2024-09-05T13:22:30.924165+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:30.924165+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1810432,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16587,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20678","reported_seq":20872,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.059015+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:31.059015+0000","last_peered":"2024-09-05T13:22:31.059015+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:31.059015+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16577,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4018}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20665","reported_seq":20803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.010757+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:31.010757+0000","last_peered":"2024-09-05T13:22:31.010757+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:31.010757+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20761","reported_seq":20907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.024100+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:31.024100+0000","last_peered":"2024-09-05T13:22:31.024100+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:31.024100+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1765859,"num_objects":432,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17429,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":29082107,"num_objects":7102,"num_object_clones":0,"num_object_copies":14204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284085,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12062,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":276791296,"data_stored":276710892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502256,"kb_used_data":159580,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93869584,"statfs":{"total":96636764160,"available":96122454016,"internally_reserved":0,"allocated":163409920,"data_stored":163139918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565656,"kb_used_data":92552,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93806184,"statfs":{"total":96636764160,"available":96057532416,"internally_reserved":0,"allocated":94773248,"data_stored":94503241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,7,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494532,"kb_used_data":21364,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93877308,"statfs":{"total":96636764160,"available":96130363392,"internally_reserved":0,"allocated":21876736,"data_stored":21541236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93364224,"data_stored":93360611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21311488,"data_stored":21242382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162115584,"data_stored":162107899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:35.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:36.045 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:36.047 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:36.077 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-05T13:22:35.744872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":24814094,"num_objects":6062,"num_object_clones":0,"num_object_copies":12124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":285129,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12167,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1546708,"kb_used_data":261916,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281568812,"statfs":{"total":289910292480,"available":288326463488,"internally_reserved":0,"allocated":268201984,"data_stored":267326475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1315556358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,9,9,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23441408,"num_objects":-5723,"num_object_clones":0,"num_object_copies":-11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-328,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5723,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005533"},"pg_stats":[{"pgid":"2.f","version":"47'20526","reported_seq":20607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.067745+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:31.067745+0000","last_peered":"2024-09-05T13:22:31.067745+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:31.067745+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1826816,"num_objects":446,"num_object_clones":0,"num_object_copies":892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20526,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3975}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20422","reported_seq":20533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:34.007307+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:34.007307+0000","last_peered":"2024-09-05T13:22:34.007307+0000","last_clean":"2024-09-05T13:22:34.007307+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:34.007307+0000","last_undegraded":"2024-09-05T13:22:34.007307+0000","last_fullsized":"2024-09-05T13:22:34.007307+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20871","reported_seq":27679,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:33.994050+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:33.994050+0000","last_peered":"2024-09-05T13:22:33.994050+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:33.994050+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1327128,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18682,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-832}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20677","reported_seq":20752,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:34.030303+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:34.030303+0000","last_peered":"2024-09-05T13:22:34.030303+0000","last_clean":"2024-09-05T13:22:34.030303+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:34.030303+0000","last_undegraded":"2024-09-05T13:22:34.030303+0000","last_fullsized":"2024-09-05T13:22:34.030303+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20677,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20447","reported_seq":20588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:31.093911+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:31.093911+0000","last_peered":"2024-09-05T13:22:31.093911+0000","last_clean":"2024-09-05T13:22:31.093911+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:31.093911+0000","last_undegraded":"2024-09-05T13:22:31.093911+0000","last_fullsized":"2024-09-05T13:22:31.093911+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1785856,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16987,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20748","reported_seq":20893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.041097+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:31.041097+0000","last_peered":"2024-09-05T13:22:31.041097+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:31.041097+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667072,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20363","reported_seq":29395,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:35.694468+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:35.694468+0000","last_peered":"2024-09-05T13:22:35.694468+0000","last_clean":"2024-09-05T13:22:35.694468+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:35.694468+0000","last_undegraded":"2024-09-05T13:22:35.694468+0000","last_fullsized":"2024-09-05T13:22:35.694468+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085440,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16682,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20706","reported_seq":20844,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:30.942131+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:30.942131+0000","last_peered":"2024-09-05T13:22:30.942131+0000","last_clean":"2024-09-05T13:22:30.942131+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:30.942131+0000","last_undegraded":"2024-09-05T13:22:30.942131+0000","last_fullsized":"2024-09-05T13:22:30.942131+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17316,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20734","reported_seq":20875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:34.021558+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:34.021558+0000","last_peered":"2024-09-05T13:22:34.021558+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:34.021558+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470464,"num_objects":359,"num_object_clones":0,"num_object_copies":718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18388,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20490","reported_seq":20634,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.081671+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:31.081671+0000","last_peered":"2024-09-05T13:22:31.081671+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:31.081671+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1720320,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16857,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4034}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20623","reported_seq":29530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:33.998915+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:33.998915+0000","last_peered":"2024-09-05T13:22:33.998915+0000","last_clean":"2024-09-05T13:22:33.998915+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:33.998915+0000","last_undegraded":"2024-09-05T13:22:33.998915+0000","last_fullsized":"2024-09-05T13:22:33.998915+0000","mapping_epoch":44,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":288,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18218,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20262","reported_seq":20392,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:34.050381+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:34.050381+0000","last_peered":"2024-09-05T13:22:34.050381+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:34.050381+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":320,"num_object_clones":0,"num_object_copies":640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17771,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4067}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20436","reported_seq":20583,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:30.924165+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:30.924165+0000","last_peered":"2024-09-05T13:22:30.924165+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:30.924165+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1810432,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16587,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20678","reported_seq":20872,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.059015+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:31.059015+0000","last_peered":"2024-09-05T13:22:31.059015+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:31.059015+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16577,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4018}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20665","reported_seq":20803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.010757+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:31.010757+0000","last_peered":"2024-09-05T13:22:31.010757+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:31.010757+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20761","reported_seq":20907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.024100+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:31.024100+0000","last_peered":"2024-09-05T13:22:31.024100+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:31.024100+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1765859,"num_objects":432,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17429,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":24814075,"num_objects":6060,"num_object_clones":0,"num_object_copies":12120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":285127,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12166,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":264638464,"data_stored":264558060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501768,"kb_used_data":154932,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93870072,"statfs":{"total":96636764160,"available":96122953728,"internally_reserved":0,"allocated":158650368,"data_stored":158380366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565656,"kb_used_data":92552,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93806184,"statfs":{"total":96636764160,"available":96057532416,"internally_reserved":0,"allocated":94773248,"data_stored":94503241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,7,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479284,"kb_used_data":14432,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93892556,"statfs":{"total":96636764160,"available":96145977344,"internally_reserved":0,"allocated":14778368,"data_stored":14442868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93364224,"data_stored":93360611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14032896,"data_stored":13963790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157241344,"data_stored":157233659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:36.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:36.634 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:36.634 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:36.664 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-05T13:22:35.744872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":24814094,"num_objects":6062,"num_object_clones":0,"num_object_copies":12124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6062,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":285129,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12167,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1546708,"kb_used_data":261916,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281568812,"statfs":{"total":289910292480,"available":288326463488,"internally_reserved":0,"allocated":268201984,"data_stored":267326475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1315556358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,9,9,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23441408,"num_objects":-5723,"num_object_clones":0,"num_object_copies":-11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-328,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5723,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005533"},"pg_stats":[{"pgid":"2.f","version":"47'20526","reported_seq":20607,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.067745+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:31.067745+0000","last_peered":"2024-09-05T13:22:31.067745+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:31.067745+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1826816,"num_objects":446,"num_object_clones":0,"num_object_copies":892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20526,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3975}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20422","reported_seq":20533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:34.007307+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:34.007307+0000","last_peered":"2024-09-05T13:22:34.007307+0000","last_clean":"2024-09-05T13:22:34.007307+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:34.007307+0000","last_undegraded":"2024-09-05T13:22:34.007307+0000","last_fullsized":"2024-09-05T13:22:34.007307+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20871","reported_seq":27679,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:33.994050+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:33.994050+0000","last_peered":"2024-09-05T13:22:33.994050+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:33.994050+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1327128,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18682,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-832}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20677","reported_seq":20752,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:34.030303+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:34.030303+0000","last_peered":"2024-09-05T13:22:34.030303+0000","last_clean":"2024-09-05T13:22:34.030303+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:34.030303+0000","last_undegraded":"2024-09-05T13:22:34.030303+0000","last_fullsized":"2024-09-05T13:22:34.030303+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20677,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20447","reported_seq":20588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:31.093911+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:31.093911+0000","last_peered":"2024-09-05T13:22:31.093911+0000","last_clean":"2024-09-05T13:22:31.093911+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:31.093911+0000","last_undegraded":"2024-09-05T13:22:31.093911+0000","last_fullsized":"2024-09-05T13:22:31.093911+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1785856,"num_objects":436,"num_object_clones":0,"num_object_copies":872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16987,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20748","reported_seq":20893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.041097+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:31.041097+0000","last_peered":"2024-09-05T13:22:31.041097+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:31.041097+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667072,"num_objects":407,"num_object_clones":0,"num_object_copies":814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17203,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20363","reported_seq":29395,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:35.694468+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:35.694468+0000","last_peered":"2024-09-05T13:22:35.694468+0000","last_clean":"2024-09-05T13:22:35.694468+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:35.694468+0000","last_undegraded":"2024-09-05T13:22:35.694468+0000","last_fullsized":"2024-09-05T13:22:35.694468+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085440,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16682,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20706","reported_seq":20844,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:30.942131+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:30.942131+0000","last_peered":"2024-09-05T13:22:30.942131+0000","last_clean":"2024-09-05T13:22:30.942131+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:30.942131+0000","last_undegraded":"2024-09-05T13:22:30.942131+0000","last_fullsized":"2024-09-05T13:22:30.942131+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17316,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20734","reported_seq":20875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:34.021558+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:34.021558+0000","last_peered":"2024-09-05T13:22:34.021558+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:34.021558+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470464,"num_objects":359,"num_object_clones":0,"num_object_copies":718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18388,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20490","reported_seq":20634,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.081671+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:31.081671+0000","last_peered":"2024-09-05T13:22:31.081671+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:31.081671+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1720320,"num_objects":420,"num_object_clones":0,"num_object_copies":840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16857,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4034}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20623","reported_seq":29530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:33.998915+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:33.998915+0000","last_peered":"2024-09-05T13:22:33.998915+0000","last_clean":"2024-09-05T13:22:33.998915+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:33.998915+0000","last_undegraded":"2024-09-05T13:22:33.998915+0000","last_fullsized":"2024-09-05T13:22:33.998915+0000","mapping_epoch":44,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":288,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18218,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20262","reported_seq":20392,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:34.050381+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:34.050381+0000","last_peered":"2024-09-05T13:22:34.050381+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:34.050381+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":320,"num_object_clones":0,"num_object_copies":640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17771,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4067}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20436","reported_seq":20583,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:30.924165+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:30.924165+0000","last_peered":"2024-09-05T13:22:30.924165+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:30.924165+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1810432,"num_objects":442,"num_object_clones":0,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16587,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20678","reported_seq":20872,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.059015+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:31.059015+0000","last_peered":"2024-09-05T13:22:31.059015+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:31.059015+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1744896,"num_objects":426,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16577,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4018}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20665","reported_seq":20803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.010757+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:31.010757+0000","last_peered":"2024-09-05T13:22:31.010757+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:31.010757+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20761","reported_seq":20907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:31.024100+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:31.024100+0000","last_peered":"2024-09-05T13:22:31.024100+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:31.024100+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1765859,"num_objects":432,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17429,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":24814075,"num_objects":6060,"num_object_clones":0,"num_object_copies":12120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":285127,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12166,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":264638464,"data_stored":264558060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501768,"kb_used_data":154932,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93870072,"statfs":{"total":96636764160,"available":96122953728,"internally_reserved":0,"allocated":158650368,"data_stored":158380366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565656,"kb_used_data":92552,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93806184,"statfs":{"total":96636764160,"available":96057532416,"internally_reserved":0,"allocated":94773248,"data_stored":94503241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":484437674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,7,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479284,"kb_used_data":14432,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93892556,"statfs":{"total":96636764160,"available":96145977344,"internally_reserved":0,"allocated":14778368,"data_stored":14442868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93364224,"data_stored":93360611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14032896,"data_stored":13963790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157241344,"data_stored":157233659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:36.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:36.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20526", 'reported_seq': 20607, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:31.067745+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:31.067745+0000', 'last_peered': '2024-09-05T13:22:31.067745+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:31.067745+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1826816, 'num_objects': 446, 'num_object_clones': 0, 'num_object_copies': 892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20526, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3975}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:36.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:36.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'20871", 'reported_seq': 27679, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:33.994050+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:22:33.994050+0000', 'last_peered': '2024-09-05T13:22:33.994050+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:33.994050+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1327128, 'num_objects': 325, 'num_object_clones': 0, 'num_object_copies': 650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 325, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18682, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3336, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -832}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:36.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:36.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'20748", 'reported_seq': 20893, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:31.041097+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:31.041097+0000', 'last_peered': '2024-09-05T13:22:31.041097+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:31.041097+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1667072, 'num_objects': 407, 'num_object_clones': 0, 'num_object_copies': 814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17203, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4044}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'20734", 'reported_seq': 20875, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:34.021558+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:34.021558+0000', 'last_peered': '2024-09-05T13:22:34.021558+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:34.021558+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1470464, 'num_objects': 359, 'num_object_clones': 0, 'num_object_copies': 718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18388, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4092}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20490", 'reported_seq': 20634, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:31.081671+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:31.081671+0000', 'last_peered': '2024-09-05T13:22:31.081671+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:31.081671+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1720320, 'num_objects': 420, 'num_object_clones': 0, 'num_object_copies': 840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16857, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4034}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20262", 'reported_seq': 20392, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:34.050381+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:34.050381+0000', 'last_peered': '2024-09-05T13:22:34.050381+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:34.050381+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1310720, 'num_objects': 320, 'num_object_clones': 0, 'num_object_copies': 640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 320, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17771, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4067}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20436", 'reported_seq': 20583, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:30.924165+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:30.924165+0000', 'last_peered': '2024-09-05T13:22:30.924165+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:30.924165+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1810432, 'num_objects': 442, 'num_object_clones': 0, 'num_object_copies': 884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16587, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3977}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:36.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:36.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'20678", 'reported_seq': 20872, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:31.059015+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:31.059015+0000', 'last_peered': '2024-09-05T13:22:31.059015+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:31.059015+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1744896, 'num_objects': 426, 'num_object_clones': 0, 'num_object_copies': 852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16577, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4018}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:36.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:36.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'20665", 'reported_seq': 20803, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:31.010757+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:31.010757+0000', 'last_peered': '2024-09-05T13:22:31.010757+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:31.010757+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1699840, 'num_objects': 415, 'num_object_clones': 0, 'num_object_copies': 830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16795, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4151}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:36.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:36.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'20761", 'reported_seq': 20907, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:31.024100+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:31.024100+0000', 'last_peered': '2024-09-05T13:22:31.024100+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:31.024100+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1765859, 'num_objects': 432, 'num_object_clones': 0, 'num_object_copies': 864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17429, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4133}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:36.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:37.365 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:37.365 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41332/12124 objects degraded (340.911%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":684,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":6062,"data_bytes":24814094,"bytes_used":1583828992,"bytes_avail":288326463488,"bytes_total":289910292480,"degraded_objects":41332,"degraded_total":12124,"degraded_ratio":3.4091059056417019,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":328,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":476},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:37.389 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:37.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:37.983 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:37.985 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:38.014 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-05T13:22:37.746941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19382798,"num_objects":4736,"num_object_clones":0,"num_object_copies":9472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286455,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12167,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160838,"ondisk_log_size":160838,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1549308,"kb_used_data":254784,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281566212,"statfs":{"total":289910292480,"available":288323801088,"internally_reserved":0,"allocated":260898816,"data_stored":260023307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1325517830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,10,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24866816,"num_objects":-6071,"num_object_clones":0,"num_object_copies":-12142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007001"},"pg_stats":[{"pgid":"2.f","version":"47'20692","reported_seq":20773,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.141076+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:36.141076+0000","last_peered":"2024-09-05T13:22:36.141076+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:36.141076+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1146880,"num_objects":280,"num_object_clones":0,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20692,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4141}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20422","reported_seq":20533,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:34.007307+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:34.007307+0000","last_peered":"2024-09-05T13:22:34.007307+0000","last_clean":"2024-09-05T13:22:34.007307+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:34.007307+0000","last_undegraded":"2024-09-05T13:22:34.007307+0000","last_fullsized":"2024-09-05T13:22:34.007307+0000","mapping_epoch":35,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1359872,"num_objects":332,"num_object_clones":0,"num_object_copies":664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20871","reported_seq":27679,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:33.994050+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:33.994050+0000","last_peered":"2024-09-05T13:22:33.994050+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:33.994050+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1327128,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18682,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3336,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-832}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20677","reported_seq":20752,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:34.030303+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:34.030303+0000","last_peered":"2024-09-05T13:22:34.030303+0000","last_clean":"2024-09-05T13:22:34.030303+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:34.030303+0000","last_undegraded":"2024-09-05T13:22:34.030303+0000","last_fullsized":"2024-09-05T13:22:34.030303+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429504,"num_objects":349,"num_object_clones":0,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20677,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20595","reported_seq":20736,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:36.072619+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:36.072619+0000","last_peered":"2024-09-05T13:22:36.072619+0000","last_clean":"2024-09-05T13:22:36.072619+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:36.072619+0000","last_undegraded":"2024-09-05T13:22:36.072619+0000","last_fullsized":"2024-09-05T13:22:36.072619+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":288,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20879","reported_seq":21024,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.060874+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:36.060874+0000","last_peered":"2024-09-05T13:22:36.060874+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:36.060874+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1130496,"num_objects":276,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17334,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4175}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20363","reported_seq":29395,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:35.694468+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:35.694468+0000","last_peered":"2024-09-05T13:22:35.694468+0000","last_clean":"2024-09-05T13:22:35.694468+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:35.694468+0000","last_undegraded":"2024-09-05T13:22:35.694468+0000","last_fullsized":"2024-09-05T13:22:35.694468+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085440,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16682,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20851","reported_seq":20989,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:36.053081+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:36.053081+0000","last_peered":"2024-09-05T13:22:36.053081+0000","last_clean":"2024-09-05T13:22:36.053081+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:36.053081+0000","last_undegraded":"2024-09-05T13:22:36.053081+0000","last_fullsized":"2024-09-05T13:22:36.053081+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036288,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17461,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20734","reported_seq":20875,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:34.021558+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:34.021558+0000","last_peered":"2024-09-05T13:22:34.021558+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:34.021558+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470464,"num_objects":359,"num_object_clones":0,"num_object_copies":718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18388,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20623","reported_seq":20768,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.080101+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:36.080101+0000","last_peered":"2024-09-05T13:22:36.080101+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:36.080101+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175552,"num_objects":287,"num_object_clones":0,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16990,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20623","reported_seq":29530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:33.998915+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:33.998915+0000","last_peered":"2024-09-05T13:22:33.998915+0000","last_clean":"2024-09-05T13:22:33.998915+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:33.998915+0000","last_undegraded":"2024-09-05T13:22:33.998915+0000","last_fullsized":"2024-09-05T13:22:33.998915+0000","mapping_epoch":44,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":288,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18218,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20262","reported_seq":20392,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:34.050381+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:34.050381+0000","last_peered":"2024-09-05T13:22:34.050381+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:34.050381+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1310720,"num_objects":320,"num_object_clones":0,"num_object_copies":640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17771,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4067}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20589","reported_seq":20736,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:35.991863+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:35.991863+0000","last_peered":"2024-09-05T13:22:35.991863+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:35.991863+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183744,"num_objects":289,"num_object_clones":0,"num_object_copies":578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16740,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20820","reported_seq":21014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.094987+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:36.094987+0000","last_peered":"2024-09-05T13:22:36.094987+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:36.094987+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1163264,"num_objects":284,"num_object_clones":0,"num_object_copies":568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16719,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4160}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20823","reported_seq":20968,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.087851+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:36.087851+0000","last_peered":"2024-09-05T13:22:36.087851+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:36.087851+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052672,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16953,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20911","reported_seq":21057,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.015934+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:36.015934+0000","last_peered":"2024-09-05T13:22:36.015934+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:36.015934+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1151459,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17579,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":19382779,"num_objects":4734,"num_object_clones":0,"num_object_copies":9468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4734,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286453,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12166,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":257355776,"data_stored":257275372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160835,"ondisk_log_size":160835,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501768,"kb_used_data":154932,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93870072,"statfs":{"total":96636764160,"available":96122953728,"internally_reserved":0,"allocated":158650368,"data_stored":158380366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":355135149},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568256,"kb_used_data":85420,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93803584,"statfs":{"total":96636764160,"available":96054870016,"internally_reserved":0,"allocated":87470080,"data_stored":87200073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":494399146},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479284,"kb_used_data":14432,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93892556,"statfs":{"total":96636764160,"available":96145977344,"internally_reserved":0,"allocated":14778368,"data_stored":14442868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86081536,"data_stored":86077923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14032896,"data_stored":13963790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157241344,"data_stored":157233659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:41.017 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:41.550 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:41.553 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:41.575 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-05T13:22:39.747396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15946254,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":287294,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12284,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160777,"ondisk_log_size":160777,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532264,"kb_used_data":250224,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281583256,"statfs":{"total":289910292480,"available":288341254144,"internally_reserved":0,"allocated":256229376,"data_stored":255353867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1312738310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,6,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22675456,"num_objects":-5536,"num_object_clones":0,"num_object_copies":-11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005292"},"pg_stats":[{"pgid":"2.f","version":"47'20692","reported_seq":20773,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.141076+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:36.141076+0000","last_peered":"2024-09-05T13:22:36.141076+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:36.141076+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1146880,"num_objects":280,"num_object_clones":0,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20692,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4141}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20565","reported_seq":20676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.081769+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:39.081769+0000","last_peered":"2024-09-05T13:22:39.081769+0000","last_clean":"2024-09-05T13:22:39.081769+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:39.081769+0000","last_undegraded":"2024-09-05T13:22:39.081769+0000","last_fullsized":"2024-09-05T13:22:39.081769+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18575,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21020","reported_seq":28062,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.081009+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:39.081009+0000","last_peered":"2024-09-05T13:22:39.081009+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:39.081009+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716824,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18831,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-864}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20822","reported_seq":20898,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.074024+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:39.074024+0000","last_peered":"2024-09-05T13:22:39.074024+0000","last_clean":"2024-09-05T13:22:39.074024+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:39.074024+0000","last_undegraded":"2024-09-05T13:22:39.074024+0000","last_fullsized":"2024-09-05T13:22:39.074024+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":835584,"num_objects":204,"num_object_clones":0,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20822,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20595","reported_seq":20736,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:36.072619+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:36.072619+0000","last_peered":"2024-09-05T13:22:36.072619+0000","last_clean":"2024-09-05T13:22:36.072619+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:36.072619+0000","last_undegraded":"2024-09-05T13:22:36.072619+0000","last_fullsized":"2024-09-05T13:22:36.072619+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1179648,"num_objects":288,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20879","reported_seq":21024,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.060874+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:36.060874+0000","last_peered":"2024-09-05T13:22:36.060874+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:36.060874+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1130496,"num_objects":276,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17334,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4175}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20363","reported_seq":29395,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:35.694468+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:35.694468+0000","last_peered":"2024-09-05T13:22:35.694468+0000","last_clean":"2024-09-05T13:22:35.694468+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:35.694468+0000","last_undegraded":"2024-09-05T13:22:35.694468+0000","last_fullsized":"2024-09-05T13:22:35.694468+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085440,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16682,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20851","reported_seq":20989,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:36.053081+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:36.053081+0000","last_peered":"2024-09-05T13:22:36.053081+0000","last_clean":"2024-09-05T13:22:36.053081+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:36.053081+0000","last_undegraded":"2024-09-05T13:22:36.053081+0000","last_fullsized":"2024-09-05T13:22:36.053081+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036288,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17461,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20881","reported_seq":21022,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.039153+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:39.039153+0000","last_peered":"2024-09-05T13:22:39.039153+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:39.039153+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":868352,"num_objects":212,"num_object_clones":0,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18535,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20623","reported_seq":20768,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.080101+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:36.080101+0000","last_peered":"2024-09-05T13:22:36.080101+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:36.080101+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175552,"num_objects":287,"num_object_clones":0,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16990,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20739","reported_seq":29646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.039701+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:39.039701+0000","last_peered":"2024-09-05T13:22:39.039701+0000","last_clean":"2024-09-05T13:22:39.039701+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:39.039701+0000","last_undegraded":"2024-09-05T13:22:39.039701+0000","last_fullsized":"2024-09-05T13:22:39.039701+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":704512,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18334,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20401","reported_seq":20531,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.026602+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:39.026602+0000","last_peered":"2024-09-05T13:22:39.026602+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:39.026602+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17910,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20589","reported_seq":20736,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:35.991863+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:35.991863+0000","last_peered":"2024-09-05T13:22:35.991863+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:35.991863+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183744,"num_objects":289,"num_object_clones":0,"num_object_copies":578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16740,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20820","reported_seq":21014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.094987+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:36.094987+0000","last_peered":"2024-09-05T13:22:36.094987+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:36.094987+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1163264,"num_objects":284,"num_object_clones":0,"num_object_copies":568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16719,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4160}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20823","reported_seq":20968,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.087851+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:36.087851+0000","last_peered":"2024-09-05T13:22:36.087851+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:36.087851+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052672,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16953,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20911","reported_seq":21057,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:36.015934+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:36.015934+0000","last_peered":"2024-09-05T13:22:36.015934+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:36.015934+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1151459,"num_objects":282,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17579,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":15946235,"num_objects":3895,"num_object_clones":0,"num_object_copies":7790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":287292,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12283,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":252710912,"data_stored":252630508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484724,"kb_used_data":150372,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93887116,"statfs":{"total":96636764160,"available":96140406784,"internally_reserved":0,"allocated":153980928,"data_stored":153710926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568256,"kb_used_data":85420,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93803584,"statfs":{"total":96636764160,"available":96054870016,"internally_reserved":0,"allocated":87470080,"data_stored":87200073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":494399146},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479284,"kb_used_data":14432,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93892556,"statfs":{"total":96636764160,"available":96145977344,"internally_reserved":0,"allocated":14778368,"data_stored":14442868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86081536,"data_stored":86077923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14032896,"data_stored":13963790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":152596480,"data_stored":152588795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:41.577 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:42.124 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:42.124 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:42.155 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-05T13:22:41.748012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9941518,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":288760,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12284,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526652,"kb_used_data":236300,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281588868,"statfs":{"total":289910292480,"available":288347000832,"internally_reserved":0,"allocated":241971200,"data_stored":241095691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1321257990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,9,7,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25116672,"num_objects":-6132,"num_object_clones":0,"num_object_copies":-12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6132,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005103"},"pg_stats":[{"pgid":"2.f","version":"47'20844","reported_seq":20922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.103546+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:41.103546+0000","last_peered":"2024-09-05T13:22:41.103546+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:41.103546+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20844,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4293}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20565","reported_seq":20676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.081769+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:39.081769+0000","last_peered":"2024-09-05T13:22:39.081769+0000","last_clean":"2024-09-05T13:22:39.081769+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:39.081769+0000","last_undegraded":"2024-09-05T13:22:39.081769+0000","last_fullsized":"2024-09-05T13:22:39.081769+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18575,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21020","reported_seq":28062,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.081009+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:39.081009+0000","last_peered":"2024-09-05T13:22:39.081009+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:39.081009+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716824,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18831,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-864}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20822","reported_seq":20898,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.074024+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:39.074024+0000","last_peered":"2024-09-05T13:22:39.074024+0000","last_clean":"2024-09-05T13:22:39.074024+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:39.074024+0000","last_undegraded":"2024-09-05T13:22:39.074024+0000","last_fullsized":"2024-09-05T13:22:39.074024+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":835584,"num_objects":204,"num_object_clones":0,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20822,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20756","reported_seq":20896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:41.106095+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:41.106095+0000","last_peered":"2024-09-05T13:22:41.106095+0000","last_clean":"2024-09-05T13:22:41.106095+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:41.106095+0000","last_undegraded":"2024-09-05T13:22:41.106095+0000","last_fullsized":"2024-09-05T13:22:41.106095+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17296,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21019","reported_seq":21164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.131114+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:41.131114+0000","last_peered":"2024-09-05T13:22:41.131114+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:41.131114+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557056,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17474,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4315}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20493","reported_seq":29526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:40.578760+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:40.578760+0000","last_peered":"2024-09-05T13:22:40.578760+0000","last_clean":"2024-09-05T13:22:40.578760+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:40.578760+0000","last_undegraded":"2024-09-05T13:22:40.578760+0000","last_fullsized":"2024-09-05T13:22:40.578760+0000","mapping_epoch":46,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":552960,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16812,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20977","reported_seq":21115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:41.158275+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:41.158275+0000","last_peered":"2024-09-05T13:22:41.158275+0000","last_clean":"2024-09-05T13:22:41.158275+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:41.158275+0000","last_undegraded":"2024-09-05T13:22:41.158275+0000","last_fullsized":"2024-09-05T13:22:41.158275+0000","mapping_epoch":34,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17587,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20881","reported_seq":21022,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.039153+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:39.039153+0000","last_peered":"2024-09-05T13:22:39.039153+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:39.039153+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":868352,"num_objects":212,"num_object_clones":0,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18535,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20779","reported_seq":20924,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.062216+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:41.062216+0000","last_peered":"2024-09-05T13:22:41.062216+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:41.062216+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536576,"num_objects":131,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17146,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4323}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20739","reported_seq":29646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.039701+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:39.039701+0000","last_peered":"2024-09-05T13:22:39.039701+0000","last_clean":"2024-09-05T13:22:39.039701+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:39.039701+0000","last_undegraded":"2024-09-05T13:22:39.039701+0000","last_fullsized":"2024-09-05T13:22:39.039701+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":704512,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18334,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20401","reported_seq":20531,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.026602+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:39.026602+0000","last_peered":"2024-09-05T13:22:39.026602+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:39.026602+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17910,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20752","reported_seq":20898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.083124+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:41.083124+0000","last_peered":"2024-09-05T13:22:41.083124+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:41.083124+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516096,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16903,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4293}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20964","reported_seq":21158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.147944+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:41.147944+0000","last_peered":"2024-09-05T13:22:41.147944+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:41.147944+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":573440,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16863,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20957","reported_seq":21103,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.158722+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:41.158722+0000","last_peered":"2024-09-05T13:22:41.158722+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:41.158722+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":503808,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17087,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21071","reported_seq":21217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.119922+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:41.119922+0000","last_peered":"2024-09-05T13:22:41.119922+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:41.119922+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":496099,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17739,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":9941499,"num_objects":2429,"num_object_clones":0,"num_object_copies":4858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":288758,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12283,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238317568,"data_stored":238237164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484724,"kb_used_data":150372,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93887116,"statfs":{"total":96636764160,"available":96140406784,"internally_reserved":0,"allocated":153980928,"data_stored":153710926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565452,"kb_used_data":78460,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93806388,"statfs":{"total":96636764160,"available":96057741312,"internally_reserved":0,"allocated":80343040,"data_stored":80073033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":498658986},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476476,"kb_used_data":7468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93895364,"statfs":{"total":96636764160,"available":96148852736,"internally_reserved":0,"allocated":7647232,"data_stored":7311732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":78790656,"data_stored":78787043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":6861326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":152596480,"data_stored":152588795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:42.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:42.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20844", 'reported_seq': 20922, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.103546+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:41.103546+0000', 'last_peered': '2024-09-05T13:22:41.103546+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:41.103546+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 524288, 'num_objects': 128, 'num_object_clones': 0, 'num_object_copies': 256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20844, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4293}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:42.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:42.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'21020", 'reported_seq': 28062, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:39.081009+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:22:39.081009+0000', 'last_peered': '2024-09-05T13:22:39.081009+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:39.081009+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 716824, 'num_objects': 176, 'num_object_clones': 0, 'num_object_copies': 352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1040, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 176, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18831, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3453, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -864}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21019", 'reported_seq': 21164, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.131114+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:41.131114+0000', 'last_peered': '2024-09-05T13:22:41.131114+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:41.131114+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 557056, 'num_objects': 136, 'num_object_clones': 0, 'num_object_copies': 272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17474, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4315}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'20881", 'reported_seq': 21022, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:39.039153+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:39.039153+0000', 'last_peered': '2024-09-05T13:22:39.039153+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:39.039153+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 868352, 'num_objects': 212, 'num_object_clones': 0, 'num_object_copies': 424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 212, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18535, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20779", 'reported_seq': 20924, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.062216+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:41.062216+0000', 'last_peered': '2024-09-05T13:22:41.062216+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:41.062216+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 536576, 'num_objects': 131, 'num_object_clones': 0, 'num_object_copies': 262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17146, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4323}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20401", 'reported_seq': 20531, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:39.026602+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:39.026602+0000', 'last_peered': '2024-09-05T13:22:39.026602+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:39.026602+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 741376, 'num_objects': 181, 'num_object_clones': 0, 'num_object_copies': 362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17910, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4206}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:42.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:42.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20752", 'reported_seq': 20898, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.083124+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:41.083124+0000', 'last_peered': '2024-09-05T13:22:41.083124+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:41.083124+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 516096, 'num_objects': 126, 'num_object_clones': 0, 'num_object_copies': 252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16903, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4293}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:42.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:42.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'20964", 'reported_seq': 21158, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.147944+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:41.147944+0000', 'last_peered': '2024-09-05T13:22:41.147944+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:41.147944+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 573440, 'num_objects': 140, 'num_object_clones': 0, 'num_object_copies': 280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 140, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16863, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:42.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:42.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'20957", 'reported_seq': 21103, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.158722+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:41.158722+0000', 'last_peered': '2024-09-05T13:22:41.158722+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:41.158722+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 503808, 'num_objects': 123, 'num_object_clones': 0, 'num_object_copies': 246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17087, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4443}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:42.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:42.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21071", 'reported_seq': 21217, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.119922+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:41.119922+0000', 'last_peered': '2024-09-05T13:22:41.119922+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:41.119922+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 496099, 'num_objects': 122, 'num_object_clones': 0, 'num_object_copies': 244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17739, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4443}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:42.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:42.883 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:42.883 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41215/4862 objects degraded (847.696%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":689,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":2431,"data_bytes":9941518,"bytes_used":1563291648,"bytes_avail":288347000832,"bytes_total":289910292480,"degraded_objects":41215,"degraded_total":4862,"degraded_ratio":8.4769642122583306,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":510},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:42.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:42.909 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:43.488 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:43.488 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:43.520 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-05T13:22:41.748012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9941518,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":288760,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12284,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526652,"kb_used_data":236300,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281588868,"statfs":{"total":289910292480,"available":288347000832,"internally_reserved":0,"allocated":241971200,"data_stored":241095691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1321257990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,9,7,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25116672,"num_objects":-6132,"num_object_clones":0,"num_object_copies":-12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6132,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005103"},"pg_stats":[{"pgid":"2.f","version":"47'20844","reported_seq":20922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.103546+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:41.103546+0000","last_peered":"2024-09-05T13:22:41.103546+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:41.103546+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20844,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4293}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20565","reported_seq":20676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.081769+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:39.081769+0000","last_peered":"2024-09-05T13:22:39.081769+0000","last_clean":"2024-09-05T13:22:39.081769+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:39.081769+0000","last_undegraded":"2024-09-05T13:22:39.081769+0000","last_fullsized":"2024-09-05T13:22:39.081769+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":774144,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18575,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21020","reported_seq":28062,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.081009+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:39.081009+0000","last_peered":"2024-09-05T13:22:39.081009+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:39.081009+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716824,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18831,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-864}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20822","reported_seq":20898,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.074024+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:39.074024+0000","last_peered":"2024-09-05T13:22:39.074024+0000","last_clean":"2024-09-05T13:22:39.074024+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:39.074024+0000","last_undegraded":"2024-09-05T13:22:39.074024+0000","last_fullsized":"2024-09-05T13:22:39.074024+0000","mapping_epoch":35,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":835584,"num_objects":204,"num_object_clones":0,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20822,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20756","reported_seq":20896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:41.106095+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:41.106095+0000","last_peered":"2024-09-05T13:22:41.106095+0000","last_clean":"2024-09-05T13:22:41.106095+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:41.106095+0000","last_undegraded":"2024-09-05T13:22:41.106095+0000","last_fullsized":"2024-09-05T13:22:41.106095+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17296,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21019","reported_seq":21164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.131114+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:41.131114+0000","last_peered":"2024-09-05T13:22:41.131114+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:41.131114+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557056,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17474,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4315}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20493","reported_seq":29526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:40.578760+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:40.578760+0000","last_peered":"2024-09-05T13:22:40.578760+0000","last_clean":"2024-09-05T13:22:40.578760+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:40.578760+0000","last_undegraded":"2024-09-05T13:22:40.578760+0000","last_fullsized":"2024-09-05T13:22:40.578760+0000","mapping_epoch":46,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":552960,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16812,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20977","reported_seq":21115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:41.158275+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:41.158275+0000","last_peered":"2024-09-05T13:22:41.158275+0000","last_clean":"2024-09-05T13:22:41.158275+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:41.158275+0000","last_undegraded":"2024-09-05T13:22:41.158275+0000","last_fullsized":"2024-09-05T13:22:41.158275+0000","mapping_epoch":34,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17587,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'20881","reported_seq":21022,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.039153+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:39.039153+0000","last_peered":"2024-09-05T13:22:39.039153+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:39.039153+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":868352,"num_objects":212,"num_object_clones":0,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18535,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4239}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119284+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:19:36.119284+0000","last_peered":"2024-09-05T13:19:36.119284+0000","last_clean":"2024-09-05T13:19:36.119284+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:19:36.119284+0000","last_undegraded":"2024-09-05T13:19:36.119284+0000","last_fullsized":"2024-09-05T13:19:36.119284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20779","reported_seq":20924,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.062216+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:41.062216+0000","last_peered":"2024-09-05T13:22:41.062216+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:41.062216+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536576,"num_objects":131,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17146,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4323}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20739","reported_seq":29646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:39.039701+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:39.039701+0000","last_peered":"2024-09-05T13:22:39.039701+0000","last_clean":"2024-09-05T13:22:39.039701+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:39.039701+0000","last_undegraded":"2024-09-05T13:22:39.039701+0000","last_fullsized":"2024-09-05T13:22:39.039701+0000","mapping_epoch":44,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":704512,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18334,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20401","reported_seq":20531,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:39.026602+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:39.026602+0000","last_peered":"2024-09-05T13:22:39.026602+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:39.026602+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17910,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20752","reported_seq":20898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.083124+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:41.083124+0000","last_peered":"2024-09-05T13:22:41.083124+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:41.083124+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516096,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16903,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4293}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20964","reported_seq":21158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.147944+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:41.147944+0000","last_peered":"2024-09-05T13:22:41.147944+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:41.147944+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":573440,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16863,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.119970+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:19:36.119970+0000","last_peered":"2024-09-05T13:19:36.119970+0000","last_clean":"2024-09-05T13:19:36.119970+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:19:36.119970+0000","last_undegraded":"2024-09-05T13:19:36.119970+0000","last_fullsized":"2024-09-05T13:19:36.119970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20957","reported_seq":21103,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.158722+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:41.158722+0000","last_peered":"2024-09-05T13:22:41.158722+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:41.158722+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":503808,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17087,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21071","reported_seq":21217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.119922+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:41.119922+0000","last_peered":"2024-09-05T13:22:41.119922+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:41.119922+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":496099,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17739,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":9941499,"num_objects":2429,"num_object_clones":0,"num_object_copies":4858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2429,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":288758,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12283,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":238317568,"data_stored":238237164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484724,"kb_used_data":150372,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93887116,"statfs":{"total":96636764160,"available":96140406784,"internally_reserved":0,"allocated":153980928,"data_stored":153710926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":342355629},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565452,"kb_used_data":78460,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93806388,"statfs":{"total":96636764160,"available":96057741312,"internally_reserved":0,"allocated":80343040,"data_stored":80073033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":498658986},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476476,"kb_used_data":7468,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93895364,"statfs":{"total":96636764160,"available":96148852736,"internally_reserved":0,"allocated":7647232,"data_stored":7311732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":78790656,"data_stored":78787043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":6861326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":152596480,"data_stored":152588795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:45.625 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Removed 167692 objects 2024-09-05T13:22:45.625 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi086.stdout:Clean up completed and total clean up time :372.364 2024-09-05T13:22:46.522 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:46.844 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:46.857 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-05T13:22:45.749747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5911054,"num_objects":1447,"num_object_clones":0,"num_object_copies":2894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289744,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12411,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1510876,"kb_used_data":224684,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281604644,"statfs":{"total":289910292480,"available":288363155456,"internally_reserved":0,"allocated":230076416,"data_stored":229204979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1316998150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,11,9,15,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23171072,"num_objects":-5657,"num_object_clones":0,"num_object_copies":-11314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-348,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5657,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005622"},"pg_stats":[{"pgid":"2.f","version":"47'20844","reported_seq":20922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.103546+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:41.103546+0000","last_peered":"2024-09-05T13:22:41.103546+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:41.103546+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20844,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4293}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20714","reported_seq":20825,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.034916+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:44.034916+0000","last_peered":"2024-09-05T13:22:44.034916+0000","last_clean":"2024-09-05T13:22:44.034916+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:44.034916+0000","last_undegraded":"2024-09-05T13:22:44.034916+0000","last_fullsized":"2024-09-05T13:22:44.034916+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163840,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21151","reported_seq":28447,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.095836+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:44.095836+0000","last_peered":"2024-09-05T13:22:44.095836+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:44.095836+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180248,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18962,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20969","reported_seq":21045,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.052330+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:44.052330+0000","last_peered":"2024-09-05T13:22:44.052330+0000","last_clean":"2024-09-05T13:22:44.052330+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:44.052330+0000","last_undegraded":"2024-09-05T13:22:44.052330+0000","last_fullsized":"2024-09-05T13:22:44.052330+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233472,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20969,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20756","reported_seq":20896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:41.106095+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:41.106095+0000","last_peered":"2024-09-05T13:22:41.106095+0000","last_clean":"2024-09-05T13:22:41.106095+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:41.106095+0000","last_undegraded":"2024-09-05T13:22:41.106095+0000","last_fullsized":"2024-09-05T13:22:41.106095+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17296,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21019","reported_seq":21164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.131114+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:41.131114+0000","last_peered":"2024-09-05T13:22:41.131114+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:41.131114+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557056,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17474,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4315}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29661,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.504937+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:45.504937+0000","last_peered":"2024-09-05T13:22:45.504937+0000","last_clean":"2024-09-05T13:22:45.504937+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:45.504937+0000","last_undegraded":"2024-09-05T13:22:45.504937+0000","last_fullsized":"2024-09-05T13:22:45.504937+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20977","reported_seq":21115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:41.158275+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:41.158275+0000","last_peered":"2024-09-05T13:22:41.158275+0000","last_clean":"2024-09-05T13:22:41.158275+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:41.158275+0000","last_undegraded":"2024-09-05T13:22:41.158275+0000","last_fullsized":"2024-09-05T13:22:41.158275+0000","mapping_epoch":34,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17587,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21041","reported_seq":21182,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.085004+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:44.085004+0000","last_peered":"2024-09-05T13:22:44.085004+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:44.085004+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212992,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18695,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719893+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:20:38.719893+0000","last_peered":"2024-09-05T13:20:38.719893+0000","last_clean":"2024-09-05T13:20:38.719893+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:20:38.719893+0000","last_undegraded":"2024-09-05T13:20:38.719893+0000","last_fullsized":"2024-09-05T13:20:38.719893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20779","reported_seq":20924,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.062216+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:41.062216+0000","last_peered":"2024-09-05T13:22:41.062216+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:41.062216+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536576,"num_objects":131,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17146,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4323}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20869","reported_seq":29775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.106046+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:44.106046+0000","last_peered":"2024-09-05T13:22:44.106046+0000","last_clean":"2024-09-05T13:22:44.106046+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:44.106046+0000","last_undegraded":"2024-09-05T13:22:44.106046+0000","last_fullsized":"2024-09-05T13:22:44.106046+0000","mapping_epoch":44,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172032,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18464,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20533","reported_seq":20663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.016001+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:44.016001+0000","last_peered":"2024-09-05T13:22:44.016001+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:44.016001+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200704,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20752","reported_seq":20898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.083124+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:41.083124+0000","last_peered":"2024-09-05T13:22:41.083124+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:41.083124+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516096,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16903,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4293}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20964","reported_seq":21158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.147944+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:41.147944+0000","last_peered":"2024-09-05T13:22:41.147944+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:41.147944+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":573440,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16863,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719934+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:20:38.719934+0000","last_peered":"2024-09-05T13:20:38.719934+0000","last_clean":"2024-09-05T13:20:38.719934+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:20:38.719934+0000","last_undegraded":"2024-09-05T13:20:38.719934+0000","last_fullsized":"2024-09-05T13:20:38.719934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20957","reported_seq":21103,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.158722+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:41.158722+0000","last_peered":"2024-09-05T13:22:41.158722+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:41.158722+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":503808,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17087,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21071","reported_seq":21217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.119922+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:41.119922+0000","last_peered":"2024-09-05T13:22:41.119922+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:41.119922+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":496099,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17739,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":5911035,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289742,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12410,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":226869248,"data_stored":226792916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484276,"kb_used_data":145764,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93887564,"statfs":{"total":96636764160,"available":96140865536,"internally_reserved":0,"allocated":149262336,"data_stored":148992334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565452,"kb_used_data":78460,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93806388,"statfs":{"total":96636764160,"available":96057741312,"internally_reserved":0,"allocated":80343040,"data_stored":80073033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":498658986},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461148,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910692,"statfs":{"total":96636764160,"available":96164548608,"internally_reserved":0,"allocated":471040,"data_stored":139612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,4,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":78790656,"data_stored":78787043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148004864,"data_stored":147997179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:46.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:47.190 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:47.191 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:47.203 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-05T13:22:45.749747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5911054,"num_objects":1447,"num_object_clones":0,"num_object_copies":2894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289744,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12411,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1510876,"kb_used_data":224684,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281604644,"statfs":{"total":289910292480,"available":288363155456,"internally_reserved":0,"allocated":230076416,"data_stored":229204979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1316998150},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,11,9,15,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23171072,"num_objects":-5657,"num_object_clones":0,"num_object_copies":-11314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-348,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5657,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005622"},"pg_stats":[{"pgid":"2.f","version":"47'20844","reported_seq":20922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.103546+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:41.103546+0000","last_peered":"2024-09-05T13:22:41.103546+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:41.103546+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20844,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4293}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20714","reported_seq":20825,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.034916+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:44.034916+0000","last_peered":"2024-09-05T13:22:44.034916+0000","last_clean":"2024-09-05T13:22:44.034916+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:44.034916+0000","last_undegraded":"2024-09-05T13:22:44.034916+0000","last_fullsized":"2024-09-05T13:22:44.034916+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163840,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21151","reported_seq":28447,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.095836+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:44.095836+0000","last_peered":"2024-09-05T13:22:44.095836+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:44.095836+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180248,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18962,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20969","reported_seq":21045,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.052330+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:44.052330+0000","last_peered":"2024-09-05T13:22:44.052330+0000","last_clean":"2024-09-05T13:22:44.052330+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:44.052330+0000","last_undegraded":"2024-09-05T13:22:44.052330+0000","last_fullsized":"2024-09-05T13:22:44.052330+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233472,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20969,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20756","reported_seq":20896,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:41.106095+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:41.106095+0000","last_peered":"2024-09-05T13:22:41.106095+0000","last_clean":"2024-09-05T13:22:41.106095+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:41.106095+0000","last_undegraded":"2024-09-05T13:22:41.106095+0000","last_fullsized":"2024-09-05T13:22:41.106095+0000","mapping_epoch":34,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17296,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21019","reported_seq":21164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.131114+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:41.131114+0000","last_peered":"2024-09-05T13:22:41.131114+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:41.131114+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":557056,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17474,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4315}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29661,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.504937+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:45.504937+0000","last_peered":"2024-09-05T13:22:45.504937+0000","last_clean":"2024-09-05T13:22:45.504937+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:45.504937+0000","last_undegraded":"2024-09-05T13:22:45.504937+0000","last_fullsized":"2024-09-05T13:22:45.504937+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'20977","reported_seq":21115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:41.158275+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:41.158275+0000","last_peered":"2024-09-05T13:22:41.158275+0000","last_clean":"2024-09-05T13:22:41.158275+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:41.158275+0000","last_undegraded":"2024-09-05T13:22:41.158275+0000","last_fullsized":"2024-09-05T13:22:41.158275+0000","mapping_epoch":34,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17587,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21041","reported_seq":21182,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.085004+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:44.085004+0000","last_peered":"2024-09-05T13:22:44.085004+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:44.085004+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212992,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18695,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719893+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:20:38.719893+0000","last_peered":"2024-09-05T13:20:38.719893+0000","last_clean":"2024-09-05T13:20:38.719893+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:20:38.719893+0000","last_undegraded":"2024-09-05T13:20:38.719893+0000","last_fullsized":"2024-09-05T13:20:38.719893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20779","reported_seq":20924,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.062216+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:41.062216+0000","last_peered":"2024-09-05T13:22:41.062216+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:41.062216+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536576,"num_objects":131,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17146,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4323}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20869","reported_seq":29775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.106046+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:44.106046+0000","last_peered":"2024-09-05T13:22:44.106046+0000","last_clean":"2024-09-05T13:22:44.106046+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:44.106046+0000","last_undegraded":"2024-09-05T13:22:44.106046+0000","last_fullsized":"2024-09-05T13:22:44.106046+0000","mapping_epoch":44,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172032,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18464,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20533","reported_seq":20663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.016001+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:44.016001+0000","last_peered":"2024-09-05T13:22:44.016001+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:44.016001+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200704,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20752","reported_seq":20898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.083124+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:41.083124+0000","last_peered":"2024-09-05T13:22:41.083124+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:41.083124+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516096,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16903,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4293}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'20964","reported_seq":21158,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.147944+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:41.147944+0000","last_peered":"2024-09-05T13:22:41.147944+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:41.147944+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":573440,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16863,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719934+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:20:38.719934+0000","last_peered":"2024-09-05T13:20:38.719934+0000","last_clean":"2024-09-05T13:20:38.719934+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:20:38.719934+0000","last_undegraded":"2024-09-05T13:20:38.719934+0000","last_fullsized":"2024-09-05T13:20:38.719934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'20957","reported_seq":21103,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.158722+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:41.158722+0000","last_peered":"2024-09-05T13:22:41.158722+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:41.158722+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":503808,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17087,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21071","reported_seq":21217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:41.119922+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:41.119922+0000","last_peered":"2024-09-05T13:22:41.119922+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:41.119922+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":496099,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17739,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":5911035,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":289742,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12410,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":226869248,"data_stored":226792916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484276,"kb_used_data":145764,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93887564,"statfs":{"total":96636764160,"available":96140865536,"internally_reserved":0,"allocated":149262336,"data_stored":148992334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565452,"kb_used_data":78460,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93806388,"statfs":{"total":96636764160,"available":96057741312,"internally_reserved":0,"allocated":80343040,"data_stored":80073033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":498658986},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,4,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461148,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910692,"statfs":{"total":96636764160,"available":96164548608,"internally_reserved":0,"allocated":471040,"data_stored":139612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,4,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":78790656,"data_stored":78787043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148004864,"data_stored":147997179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:47.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:47.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20844", 'reported_seq': 20922, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.103546+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:41.103546+0000', 'last_peered': '2024-09-05T13:22:41.103546+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:41.103546+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 524288, 'num_objects': 128, 'num_object_clones': 0, 'num_object_copies': 256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20844, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4293}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:47.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:47.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'21151", 'reported_seq': 28447, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:44.095836+0000', 'last_change': '2024-09-05T13:19:34.404816+0000', 'last_active': '2024-09-05T13:22:44.095836+0000', 'last_peered': '2024-09-05T13:22:44.095836+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:44.095836+0000', 'last_undegraded': '2024-09-05T13:13:17.943393+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 180248, 'num_objects': 45, 'num_object_clones': 0, 'num_object_copies': 90, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 913, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 45, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18962, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3580, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:47.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:47.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21019", 'reported_seq': 21164, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.131114+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:41.131114+0000', 'last_peered': '2024-09-05T13:22:41.131114+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:41.131114+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 557056, 'num_objects': 136, 'num_object_clones': 0, 'num_object_copies': 272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17474, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4315}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:47.205 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21041", 'reported_seq': 21182, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:44.085004+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:44.085004+0000', 'last_peered': '2024-09-05T13:22:44.085004+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:44.085004+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 212992, 'num_objects': 52, 'num_object_clones': 0, 'num_object_copies': 104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 52, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18695, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20779", 'reported_seq': 20924, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.062216+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:41.062216+0000', 'last_peered': '2024-09-05T13:22:41.062216+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:41.062216+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 536576, 'num_objects': 131, 'num_object_clones': 0, 'num_object_copies': 262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17146, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4323}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20533", 'reported_seq': 20663, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:44.016001+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:44.016001+0000', 'last_peered': '2024-09-05T13:22:44.016001+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:44.016001+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 200704, 'num_objects': 49, 'num_object_clones': 0, 'num_object_copies': 98, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 49, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18042, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4338}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20752", 'reported_seq': 20898, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.083124+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:41.083124+0000', 'last_peered': '2024-09-05T13:22:41.083124+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:41.083124+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10700", 'ondisk_log_start': "45'10700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 516096, 'num_objects': 126, 'num_object_clones': 0, 'num_object_copies': 252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16903, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4293}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'20964", 'reported_seq': 21158, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.147944+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:41.147944+0000', 'last_peered': '2024-09-05T13:22:41.147944+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:41.147944+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 573440, 'num_objects': 140, 'num_object_clones': 0, 'num_object_copies': 280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 140, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16863, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:47.206 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'20957", 'reported_seq': 21103, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.158722+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:41.158722+0000', 'last_peered': '2024-09-05T13:22:41.158722+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:41.158722+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 503808, 'num_objects': 123, 'num_object_clones': 0, 'num_object_copies': 246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17087, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4443}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:47.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:47.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21071", 'reported_seq': 21217, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:41.119922+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:41.119922+0000', 'last_peered': '2024-09-05T13:22:41.119922+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:41.119922+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 496099, 'num_objects': 122, 'num_object_clones': 0, 'num_object_copies': 244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17739, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4443}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:47.207 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:47.629 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:47.629 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41088/2894 objects degraded (1419.765%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":694,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":1447,"data_bytes":5911054,"bytes_used":1547137024,"bytes_avail":288363155456,"bytes_total":289910292480,"degraded_objects":41088,"degraded_total":2894,"degraded_ratio":14.197650310988251,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":471},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:47.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:47.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:47.946 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:47.947 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:47.958 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-05T13:22:47.750080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1163790,"num_objects":288,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":290903,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12411,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1508080,"kb_used_data":217720,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281607440,"statfs":{"total":289910292480,"available":288366018560,"internally_reserved":0,"allocated":222945280,"data_stored":222073843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1321257990},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,6,7,9,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23650304,"num_objects":-5774,"num_object_clones":0,"num_object_copies":-11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005208"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21053,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.624357+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:45.624357+0000","last_peered":"2024-09-05T13:22:45.624357+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:45.624357+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20714","reported_seq":20825,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.034916+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:44.034916+0000","last_peered":"2024-09-05T13:22:44.034916+0000","last_clean":"2024-09-05T13:22:44.034916+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:44.034916+0000","last_undegraded":"2024-09-05T13:22:44.034916+0000","last_fullsized":"2024-09-05T13:22:44.034916+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163840,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18724,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21151","reported_seq":28447,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.095836+0000","last_change":"2024-09-05T13:19:34.404816+0000","last_active":"2024-09-05T13:22:44.095836+0000","last_peered":"2024-09-05T13:22:44.095836+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:44.095836+0000","last_undegraded":"2024-09-05T13:13:17.943393+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180248,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18962,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'20969","reported_seq":21045,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.052330+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:44.052330+0000","last_peered":"2024-09-05T13:22:44.052330+0000","last_clean":"2024-09-05T13:22:44.052330+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:44.052330+0000","last_undegraded":"2024-09-05T13:22:44.052330+0000","last_fullsized":"2024-09-05T13:22:44.052330+0000","mapping_epoch":35,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233472,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20969,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21024,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.466010+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:45.466010+0000","last_peered":"2024-09-05T13:22:45.466010+0000","last_clean":"2024-09-05T13:22:45.466010+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:45.466010+0000","last_undegraded":"2024-09-05T13:22:45.466010+0000","last_fullsized":"2024-09-05T13:22:45.466010+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21300,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.588751+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:45.588751+0000","last_peered":"2024-09-05T13:22:45.588751+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:45.588751+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29661,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.504937+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:45.504937+0000","last_peered":"2024-09-05T13:22:45.504937+0000","last_clean":"2024-09-05T13:22:45.504937+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:45.504937+0000","last_undegraded":"2024-09-05T13:22:45.504937+0000","last_fullsized":"2024-09-05T13:22:45.504937+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.502090+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:45.502090+0000","last_peered":"2024-09-05T13:22:45.502090+0000","last_clean":"2024-09-05T13:22:45.502090+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:45.502090+0000","last_undegraded":"2024-09-05T13:22:45.502090+0000","last_fullsized":"2024-09-05T13:22:45.502090+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106284+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:19:36.106284+0000","last_peered":"2024-09-05T13:19:36.106284+0000","last_clean":"2024-09-05T13:19:36.106284+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:19:36.106284+0000","last_undegraded":"2024-09-05T13:19:36.106284+0000","last_fullsized":"2024-09-05T13:19:36.106284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21041","reported_seq":21182,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.085004+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:44.085004+0000","last_peered":"2024-09-05T13:22:44.085004+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:44.085004+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212992,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18695,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719893+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:20:38.719893+0000","last_peered":"2024-09-05T13:20:38.719893+0000","last_clean":"2024-09-05T13:20:38.719893+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:20:38.719893+0000","last_undegraded":"2024-09-05T13:20:38.719893+0000","last_fullsized":"2024-09-05T13:20:38.719893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21055,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.562084+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:45.562084+0000","last_peered":"2024-09-05T13:22:45.562084+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:45.562084+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20869","reported_seq":29775,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:44.106046+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:44.106046+0000","last_peered":"2024-09-05T13:22:44.106046+0000","last_clean":"2024-09-05T13:22:44.106046+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:44.106046+0000","last_undegraded":"2024-09-05T13:22:44.106046+0000","last_fullsized":"2024-09-05T13:22:44.106046+0000","mapping_epoch":44,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172032,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18464,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.106934+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:19:36.106934+0000","last_peered":"2024-09-05T13:19:36.106934+0000","last_clean":"2024-09-05T13:19:36.106934+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:19:36.106934+0000","last_undegraded":"2024-09-05T13:19:36.106934+0000","last_fullsized":"2024-09-05T13:19:36.106934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20533","reported_seq":20663,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:44.016001+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:44.016001+0000","last_peered":"2024-09-05T13:22:44.016001+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:44.016001+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200704,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21025,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.619963+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:45.619963+0000","last_peered":"2024-09-05T13:22:45.619963+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:45.619963+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107135+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:19:36.107135+0000","last_peered":"2024-09-05T13:19:36.107135+0000","last_clean":"2024-09-05T13:19:36.107135+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:19:36.107135+0000","last_undegraded":"2024-09-05T13:19:36.107135+0000","last_fullsized":"2024-09-05T13:19:36.107135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.506407+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:45.506407+0000","last_peered":"2024-09-05T13:22:45.506407+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:45.506407+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719934+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:20:38.719934+0000","last_peered":"2024-09-05T13:20:38.719934+0000","last_clean":"2024-09-05T13:20:38.719934+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:20:38.719934+0000","last_undegraded":"2024-09-05T13:20:38.719934+0000","last_fullsized":"2024-09-05T13:20:38.719934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21226,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.549629+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:45.549629+0000","last_peered":"2024-09-05T13:22:45.549629+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:45.549629+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:36.107213+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:19:36.107213+0000","last_peered":"2024-09-05T13:19:36.107213+0000","last_clean":"2024-09-05T13:19:36.107213+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:19:36.107213+0000","last_undegraded":"2024-09-05T13:19:36.107213+0000","last_fullsized":"2024-09-05T13:19:36.107213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:19:48.336831+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:19:48.336831+0000","last_peered":"2024-09-05T13:19:48.336831+0000","last_clean":"2024-09-05T13:19:48.336831+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:19:48.336831+0000","last_undegraded":"2024-09-05T13:19:48.336831+0000","last_fullsized":"2024-09-05T13:19:48.336831+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21339,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.551179+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:45.551179+0000","last_peered":"2024-09-05T13:22:45.551179+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:45.551179+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1163771,"num_objects":286,"num_object_clones":0,"num_object_copies":572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":290901,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12410,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":220643328,"data_stored":220566996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484276,"kb_used_data":145764,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93887564,"statfs":{"total":96636764160,"available":96140865536,"internally_reserved":0,"allocated":149262336,"data_stored":148992334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":346615469},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562656,"kb_used_data":71496,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809184,"statfs":{"total":96636764160,"available":96060604416,"internally_reserved":0,"allocated":73211904,"data_stored":72941897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461148,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910692,"statfs":{"total":96636764160,"available":96164548608,"internally_reserved":0,"allocated":471040,"data_stored":139612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,4,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148004864,"data_stored":147997179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:50.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:51.254 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:51.254 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:51.266 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-05T13:22:50.892752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13324,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518192,"kb_used_data":215352,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281597328,"statfs":{"total":289910292480,"available":288355663872,"internally_reserved":0,"allocated":220520448,"data_stored":219660309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1334037510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9941016,"num_objects":-2428,"num_object_clones":0,"num_object_copies":-4856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2428,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1040,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.144740"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21053,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.624357+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:45.624357+0000","last_peered":"2024-09-05T13:22:45.624357+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:45.624357+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.560340+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:45.560340+0000","last_peered":"2024-09-05T13:22:45.560340+0000","last_clean":"2024-09-05T13:22:45.560340+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:45.560340+0000","last_undegraded":"2024-09-05T13:22:45.560340+0000","last_fullsized":"2024-09-05T13:22:45.560340+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28576,"reported_epoch":48,"state":"peering","last_fresh":"2024-09-05T13:22:49.022606+0000","last_change":"2024-09-05T13:22:50.742232+0000","last_active":"2024-09-05T13:22:50.742232+0000","last_peered":"2024-09-05T13:22:49.022606+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:49.022606+0000","last_undegraded":"2024-09-05T13:22:49.022606+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21102,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.550036+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:45.550036+0000","last_peered":"2024-09-05T13:22:45.550036+0000","last_clean":"2024-09-05T13:22:45.550036+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:45.550036+0000","last_undegraded":"2024-09-05T13:22:45.550036+0000","last_fullsized":"2024-09-05T13:22:45.550036+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21024,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.466010+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:45.466010+0000","last_peered":"2024-09-05T13:22:45.466010+0000","last_clean":"2024-09-05T13:22:45.466010+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:45.466010+0000","last_undegraded":"2024-09-05T13:22:45.466010+0000","last_fullsized":"2024-09-05T13:22:45.466010+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21300,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.588751+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:45.588751+0000","last_peered":"2024-09-05T13:22:45.588751+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:45.588751+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29663,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883289+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:49.883289+0000","last_peered":"2024-09-05T13:22:49.883289+0000","last_clean":"2024-09-05T13:22:49.883289+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:49.883289+0000","last_undegraded":"2024-09-05T13:22:49.883289+0000","last_fullsized":"2024-09-05T13:22:49.883289+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.502090+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:45.502090+0000","last_peered":"2024-09-05T13:22:45.502090+0000","last_clean":"2024-09-05T13:22:45.502090+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:45.502090+0000","last_undegraded":"2024-09-05T13:22:45.502090+0000","last_fullsized":"2024-09-05T13:22:45.502090+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.884196+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:49.884196+0000","last_peered":"2024-09-05T13:22:49.884196+0000","last_clean":"2024-09-05T13:22:49.884196+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:49.884196+0000","last_undegraded":"2024-09-05T13:22:49.884196+0000","last_fullsized":"2024-09-05T13:22:49.884196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.529601+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:45.529601+0000","last_peered":"2024-09-05T13:22:45.529601+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:45.529601+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719893+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:20:38.719893+0000","last_peered":"2024-09-05T13:20:38.719893+0000","last_clean":"2024-09-05T13:20:38.719893+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:20:38.719893+0000","last_undegraded":"2024-09-05T13:20:38.719893+0000","last_fullsized":"2024-09-05T13:20:38.719893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21055,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.562084+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:45.562084+0000","last_peered":"2024-09-05T13:22:45.562084+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:45.562084+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29819,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.502143+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:45.502143+0000","last_peered":"2024-09-05T13:22:45.502143+0000","last_clean":"2024-09-05T13:22:45.502143+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:45.502143+0000","last_undegraded":"2024-09-05T13:22:45.502143+0000","last_fullsized":"2024-09-05T13:22:45.502143+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":88,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883722+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:49.883722+0000","last_peered":"2024-09-05T13:22:49.883722+0000","last_clean":"2024-09-05T13:22:49.883722+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:49.883722+0000","last_undegraded":"2024-09-05T13:22:49.883722+0000","last_fullsized":"2024-09-05T13:22:49.883722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.483479+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:45.483479+0000","last_peered":"2024-09-05T13:22:45.483479+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:45.483479+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21025,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.619963+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:45.619963+0000","last_peered":"2024-09-05T13:22:45.619963+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:45.619963+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883834+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:49.883834+0000","last_peered":"2024-09-05T13:22:49.883834+0000","last_clean":"2024-09-05T13:22:49.883834+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:49.883834+0000","last_undegraded":"2024-09-05T13:22:49.883834+0000","last_fullsized":"2024-09-05T13:22:49.883834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.506407+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:45.506407+0000","last_peered":"2024-09-05T13:22:45.506407+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:45.506407+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719934+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:20:38.719934+0000","last_peered":"2024-09-05T13:20:38.719934+0000","last_clean":"2024-09-05T13:20:38.719934+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:20:38.719934+0000","last_undegraded":"2024-09-05T13:20:38.719934+0000","last_fullsized":"2024-09-05T13:20:38.719934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21226,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.549629+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:45.549629+0000","last_peered":"2024-09-05T13:22:45.549629+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:45.549629+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883885+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:49.883885+0000","last_peered":"2024-09-05T13:22:49.883885+0000","last_clean":"2024-09-05T13:22:49.883885+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:49.883885+0000","last_undegraded":"2024-09-05T13:22:49.883885+0000","last_fullsized":"2024-09-05T13:22:49.883885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.884038+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:49.884038+0000","last_peered":"2024-09-05T13:22:49.884038+0000","last_clean":"2024-09-05T13:22:49.884038+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:49.884038+0000","last_undegraded":"2024-09-05T13:22:49.884038+0000","last_fullsized":"2024-09-05T13:22:49.884038+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21339,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.551179+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:45.551179+0000","last_peered":"2024-09-05T13:22:45.551179+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:45.551179+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13323,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486076,"kb_used_data":143404,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885764,"statfs":{"total":96636764160,"available":96139022336,"internally_reserved":0,"allocated":146845696,"data_stored":146579766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562656,"kb_used_data":71496,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809184,"statfs":{"total":96636764160,"available":96060604416,"internally_reserved":0,"allocated":73211904,"data_stored":72941897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469460,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902380,"statfs":{"total":96636764160,"available":96156037120,"internally_reserved":0,"allocated":462848,"data_stored":138646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:51.268 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:51.575 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:51.588 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-05T13:22:50.892752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13324,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518192,"kb_used_data":215352,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281597328,"statfs":{"total":289910292480,"available":288355663872,"internally_reserved":0,"allocated":220520448,"data_stored":219660309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1334037510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9941016,"num_objects":-2428,"num_object_clones":0,"num_object_copies":-4856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2428,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1040,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.144740"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21053,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.624357+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:45.624357+0000","last_peered":"2024-09-05T13:22:45.624357+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:45.624357+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.560340+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:45.560340+0000","last_peered":"2024-09-05T13:22:45.560340+0000","last_clean":"2024-09-05T13:22:45.560340+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:45.560340+0000","last_undegraded":"2024-09-05T13:22:45.560340+0000","last_fullsized":"2024-09-05T13:22:45.560340+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28576,"reported_epoch":48,"state":"peering","last_fresh":"2024-09-05T13:22:49.022606+0000","last_change":"2024-09-05T13:22:50.742232+0000","last_active":"2024-09-05T13:22:50.742232+0000","last_peered":"2024-09-05T13:22:49.022606+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:49.022606+0000","last_undegraded":"2024-09-05T13:22:49.022606+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21102,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.550036+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:45.550036+0000","last_peered":"2024-09-05T13:22:45.550036+0000","last_clean":"2024-09-05T13:22:45.550036+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:45.550036+0000","last_undegraded":"2024-09-05T13:22:45.550036+0000","last_fullsized":"2024-09-05T13:22:45.550036+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21024,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.466010+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:45.466010+0000","last_peered":"2024-09-05T13:22:45.466010+0000","last_clean":"2024-09-05T13:22:45.466010+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:45.466010+0000","last_undegraded":"2024-09-05T13:22:45.466010+0000","last_fullsized":"2024-09-05T13:22:45.466010+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21300,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.588751+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:45.588751+0000","last_peered":"2024-09-05T13:22:45.588751+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:45.588751+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29663,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883289+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:49.883289+0000","last_peered":"2024-09-05T13:22:49.883289+0000","last_clean":"2024-09-05T13:22:49.883289+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:49.883289+0000","last_undegraded":"2024-09-05T13:22:49.883289+0000","last_fullsized":"2024-09-05T13:22:49.883289+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.502090+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:45.502090+0000","last_peered":"2024-09-05T13:22:45.502090+0000","last_clean":"2024-09-05T13:22:45.502090+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:45.502090+0000","last_undegraded":"2024-09-05T13:22:45.502090+0000","last_fullsized":"2024-09-05T13:22:45.502090+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.884196+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:49.884196+0000","last_peered":"2024-09-05T13:22:49.884196+0000","last_clean":"2024-09-05T13:22:49.884196+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:49.884196+0000","last_undegraded":"2024-09-05T13:22:49.884196+0000","last_fullsized":"2024-09-05T13:22:49.884196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.529601+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:45.529601+0000","last_peered":"2024-09-05T13:22:45.529601+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:45.529601+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719893+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:20:38.719893+0000","last_peered":"2024-09-05T13:20:38.719893+0000","last_clean":"2024-09-05T13:20:38.719893+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:20:38.719893+0000","last_undegraded":"2024-09-05T13:20:38.719893+0000","last_fullsized":"2024-09-05T13:20:38.719893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21055,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.562084+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:45.562084+0000","last_peered":"2024-09-05T13:22:45.562084+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:45.562084+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:25.150793+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:20:25.150793+0000","last_peered":"2024-09-05T13:20:25.150793+0000","last_clean":"2024-09-05T13:20:25.150793+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:20:25.150793+0000","last_undegraded":"2024-09-05T13:20:25.150793+0000","last_fullsized":"2024-09-05T13:20:25.150793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29819,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.502143+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:45.502143+0000","last_peered":"2024-09-05T13:22:45.502143+0000","last_clean":"2024-09-05T13:22:45.502143+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:45.502143+0000","last_undegraded":"2024-09-05T13:22:45.502143+0000","last_fullsized":"2024-09-05T13:22:45.502143+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":88,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883722+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:49.883722+0000","last_peered":"2024-09-05T13:22:49.883722+0000","last_clean":"2024-09-05T13:22:49.883722+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:49.883722+0000","last_undegraded":"2024-09-05T13:22:49.883722+0000","last_fullsized":"2024-09-05T13:22:49.883722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.483479+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:45.483479+0000","last_peered":"2024-09-05T13:22:45.483479+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:45.483479+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21025,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.619963+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:45.619963+0000","last_peered":"2024-09-05T13:22:45.619963+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:45.619963+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883834+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:49.883834+0000","last_peered":"2024-09-05T13:22:49.883834+0000","last_clean":"2024-09-05T13:22:49.883834+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:49.883834+0000","last_undegraded":"2024-09-05T13:22:49.883834+0000","last_fullsized":"2024-09-05T13:22:49.883834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21298,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.506407+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:45.506407+0000","last_peered":"2024-09-05T13:22:45.506407+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:45.506407+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719934+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:20:38.719934+0000","last_peered":"2024-09-05T13:20:38.719934+0000","last_clean":"2024-09-05T13:20:38.719934+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:20:38.719934+0000","last_undegraded":"2024-09-05T13:20:38.719934+0000","last_fullsized":"2024-09-05T13:20:38.719934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21226,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.549629+0000","last_change":"2024-09-05T13:13:18.476917+0000","last_active":"2024-09-05T13:22:45.549629+0000","last_peered":"2024-09-05T13:22:45.549629+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:45.549629+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883885+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:49.883885+0000","last_peered":"2024-09-05T13:22:49.883885+0000","last_clean":"2024-09-05T13:22:49.883885+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:49.883885+0000","last_undegraded":"2024-09-05T13:22:49.883885+0000","last_fullsized":"2024-09-05T13:22:49.883885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.884038+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:49.884038+0000","last_peered":"2024-09-05T13:22:49.884038+0000","last_clean":"2024-09-05T13:22:49.884038+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:49.884038+0000","last_undegraded":"2024-09-05T13:22:49.884038+0000","last_fullsized":"2024-09-05T13:22:49.884038+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21339,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.551179+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:45.551179+0000","last_peered":"2024-09-05T13:22:45.551179+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:45.551179+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13323,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486076,"kb_used_data":143404,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885764,"statfs":{"total":96636764160,"available":96139022336,"internally_reserved":0,"allocated":146845696,"data_stored":146579766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562656,"kb_used_data":71496,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809184,"statfs":{"total":96636764160,"available":96060604416,"internally_reserved":0,"allocated":73211904,"data_stored":72941897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,1,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469460,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902380,"statfs":{"total":96636764160,"available":96156037120,"internally_reserved":0,"allocated":462848,"data_stored":138646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:51.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:51.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21053, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.624357+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:45.624357+0000', 'last_peered': '2024-09-05T13:22:45.624357+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:45.624357+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:51.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-05T13:22:51.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'21196", 'reported_seq': 28576, 'reported_epoch': 48, 'state': 'peering', 'last_fresh': '2024-09-05T13:22:49.022606+0000', 'last_change': '2024-09-05T13:22:50.742232+0000', 'last_active': '2024-09-05T13:22:50.742232+0000', 'last_peered': '2024-09-05T13:22:49.022606+0000', 'last_clean': '2024-09-05T13:13:15.874937+0000', 'last_became_active': '2024-09-05T13:13:18.487145+0000', 'last_became_peered': '2024-09-05T13:13:18.487145+0000', 'last_unstale': '2024-09-05T13:22:49.022606+0000', 'last_undegraded': '2024-09-05T13:22:49.022606+0000', 'last_fullsized': '2024-09-05T13:13:17.904581+0000', 'mapping_epoch': 48, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2236", 'last_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:47.044046+0000', 'objects_scrubbed': 2185, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 835, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 19007, 'num_write_kb': 33641, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4493, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-05T13:22:51.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:51.589 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21300, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.588751+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:45.588751+0000', 'last_peered': '2024-09-05T13:22:45.588751+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:45.588751+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21234, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.529601+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:45.529601+0000', 'last_peered': '2024-09-05T13:22:45.529601+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:45.529601+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21055, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.562084+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:45.562084+0000', 'last_peered': '2024-09-05T13:22:45.562084+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:45.562084+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20712, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.483479+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:45.483479+0000', 'last_peered': '2024-09-05T13:22:45.483479+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:45.483479+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21025, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.619963+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:45.619963+0000', 'last_peered': '2024-09-05T13:22:45.619963+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:45.619963+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:51.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21298, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.506407+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:45.506407+0000', 'last_peered': '2024-09-05T13:22:45.506407+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:45.506407+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:51.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:51.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'21080", 'reported_seq': 21226, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.549629+0000', 'last_change': '2024-09-05T13:13:18.476917+0000', 'last_active': '2024-09-05T13:22:45.549629+0000', 'last_peered': '2024-09-05T13:22:45.549629+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:45.549629+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17210, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:51.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:51.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21339, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:45.551179+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:45.551179+0000', 'last_peered': '2024-09-05T13:22:45.551179+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:45.551179+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:51.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:51.979 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:51.979 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39890/6 objects degraded (664833.333%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1554382848,"bytes_avail":288355909632,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":39890,"degraded_total":6,"degraded_ratio":6648.3333333333321,"recovering_objects_per_sec":165,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1325,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":303},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:51.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:51.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:52.316 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:52.316 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:52.328 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-05T13:22:51.750751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39890,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13609,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517952,"kb_used_data":215116,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281597568,"statfs":{"total":289910292480,"available":288355909632,"internally_reserved":0,"allocated":220278784,"data_stored":219413408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1334037510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9941016,"num_objects":-2428,"num_object_clones":0,"num_object_copies":-4856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2428,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1325,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001934"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.560340+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:45.560340+0000","last_peered":"2024-09-05T13:22:45.560340+0000","last_clean":"2024-09-05T13:22:45.560340+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:45.560340+0000","last_undegraded":"2024-09-05T13:22:45.560340+0000","last_fullsized":"2024-09-05T13:22:45.560340+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28576,"reported_epoch":48,"state":"peering","last_fresh":"2024-09-05T13:22:49.022606+0000","last_change":"2024-09-05T13:22:50.742232+0000","last_active":"2024-09-05T13:22:50.742232+0000","last_peered":"2024-09-05T13:22:49.022606+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:49.022606+0000","last_undegraded":"2024-09-05T13:22:49.022606+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21102,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.550036+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:45.550036+0000","last_peered":"2024-09-05T13:22:45.550036+0000","last_clean":"2024-09-05T13:22:45.550036+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:45.550036+0000","last_undegraded":"2024-09-05T13:22:45.550036+0000","last_fullsized":"2024-09-05T13:22:45.550036+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29663,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883289+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:49.883289+0000","last_peered":"2024-09-05T13:22:49.883289+0000","last_clean":"2024-09-05T13:22:49.883289+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:49.883289+0000","last_undegraded":"2024-09-05T13:22:49.883289+0000","last_fullsized":"2024-09-05T13:22:49.883289+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21246,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896611+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896611+0000","last_peered":"2024-09-05T13:22:50.896611+0000","last_clean":"2024-09-05T13:22:50.896611+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896611+0000","last_undegraded":"2024-09-05T13:22:50.896611+0000","last_fullsized":"2024-09-05T13:22:50.896611+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.884196+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:49.884196+0000","last_peered":"2024-09-05T13:22:49.884196+0000","last_clean":"2024-09-05T13:22:49.884196+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:49.884196+0000","last_undegraded":"2024-09-05T13:22:49.884196+0000","last_fullsized":"2024-09-05T13:22:49.884196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.529601+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:45.529601+0000","last_peered":"2024-09-05T13:22:45.529601+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:45.529601+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719893+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:20:38.719893+0000","last_peered":"2024-09-05T13:20:38.719893+0000","last_clean":"2024-09-05T13:20:38.719893+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:20:38.719893+0000","last_undegraded":"2024-09-05T13:20:38.719893+0000","last_fullsized":"2024-09-05T13:20:38.719893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29819,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.502143+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:45.502143+0000","last_peered":"2024-09-05T13:22:45.502143+0000","last_clean":"2024-09-05T13:22:45.502143+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:45.502143+0000","last_undegraded":"2024-09-05T13:22:45.502143+0000","last_fullsized":"2024-09-05T13:22:45.502143+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":88,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883722+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:49.883722+0000","last_peered":"2024-09-05T13:22:49.883722+0000","last_clean":"2024-09-05T13:22:49.883722+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:49.883722+0000","last_undegraded":"2024-09-05T13:22:49.883722+0000","last_fullsized":"2024-09-05T13:22:49.883722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.483479+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:45.483479+0000","last_peered":"2024-09-05T13:22:45.483479+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:45.483479+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883834+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:49.883834+0000","last_peered":"2024-09-05T13:22:49.883834+0000","last_clean":"2024-09-05T13:22:49.883834+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:49.883834+0000","last_undegraded":"2024-09-05T13:22:49.883834+0000","last_fullsized":"2024-09-05T13:22:49.883834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719934+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:20:38.719934+0000","last_peered":"2024-09-05T13:20:38.719934+0000","last_clean":"2024-09-05T13:20:38.719934+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:20:38.719934+0000","last_undegraded":"2024-09-05T13:20:38.719934+0000","last_fullsized":"2024-09-05T13:20:38.719934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21801,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:51.180387+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:22:51.180387+0000","last_peered":"2024-09-05T13:22:51.180387+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:51.180387+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883885+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:49.883885+0000","last_peered":"2024-09-05T13:22:49.883885+0000","last_clean":"2024-09-05T13:22:49.883885+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:49.883885+0000","last_undegraded":"2024-09-05T13:22:49.883885+0000","last_fullsized":"2024-09-05T13:22:49.883885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.884038+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:49.884038+0000","last_peered":"2024-09-05T13:22:49.884038+0000","last_clean":"2024-09-05T13:22:49.884038+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:49.884038+0000","last_undegraded":"2024-09-05T13:22:49.884038+0000","last_fullsized":"2024-09-05T13:22:49.884038+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39890,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13608,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486076,"kb_used_data":143404,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885764,"statfs":{"total":96636764160,"available":96139022336,"internally_reserved":0,"allocated":146845696,"data_stored":146579766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562416,"kb_used_data":71260,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809424,"statfs":{"total":96636764160,"available":96060850176,"internally_reserved":0,"allocated":72970240,"data_stored":72694996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469460,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902380,"statfs":{"total":96636764160,"available":96156037120,"internally_reserved":0,"allocated":462848,"data_stored":138646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:55.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:55.678 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:55.679 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:55.691 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-05T13:22:53.751094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39890,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13609,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517952,"kb_used_data":215116,"kb_used_omap":12,"kb_used_meta":1302771,"kb_avail":281597568,"statfs":{"total":289910292480,"available":288355909632,"internally_reserved":0,"allocated":220278784,"data_stored":219413408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1334037510},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5910552,"num_objects":-1444,"num_object_clones":0,"num_object_copies":-2888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1444,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001346"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20865,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.560340+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:45.560340+0000","last_peered":"2024-09-05T13:22:45.560340+0000","last_clean":"2024-09-05T13:22:45.560340+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:45.560340+0000","last_undegraded":"2024-09-05T13:22:45.560340+0000","last_fullsized":"2024-09-05T13:22:45.560340+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28576,"reported_epoch":48,"state":"peering","last_fresh":"2024-09-05T13:22:49.022606+0000","last_change":"2024-09-05T13:22:50.742232+0000","last_active":"2024-09-05T13:22:50.742232+0000","last_peered":"2024-09-05T13:22:49.022606+0000","last_clean":"2024-09-05T13:13:15.874937+0000","last_became_active":"2024-09-05T13:13:18.487145+0000","last_became_peered":"2024-09-05T13:13:18.487145+0000","last_unstale":"2024-09-05T13:22:49.022606+0000","last_undegraded":"2024-09-05T13:22:49.022606+0000","last_fullsized":"2024-09-05T13:13:17.904581+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21102,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.550036+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:45.550036+0000","last_peered":"2024-09-05T13:22:45.550036+0000","last_clean":"2024-09-05T13:22:45.550036+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:45.550036+0000","last_undegraded":"2024-09-05T13:22:45.550036+0000","last_fullsized":"2024-09-05T13:22:45.550036+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29663,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883289+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:49.883289+0000","last_peered":"2024-09-05T13:22:49.883289+0000","last_clean":"2024-09-05T13:22:49.883289+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:49.883289+0000","last_undegraded":"2024-09-05T13:22:49.883289+0000","last_fullsized":"2024-09-05T13:22:49.883289+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21246,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896611+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896611+0000","last_peered":"2024-09-05T13:22:50.896611+0000","last_clean":"2024-09-05T13:22:50.896611+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896611+0000","last_undegraded":"2024-09-05T13:22:50.896611+0000","last_fullsized":"2024-09-05T13:22:50.896611+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.884196+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:49.884196+0000","last_peered":"2024-09-05T13:22:49.884196+0000","last_clean":"2024-09-05T13:22:49.884196+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:49.884196+0000","last_undegraded":"2024-09-05T13:22:49.884196+0000","last_fullsized":"2024-09-05T13:22:49.884196+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.529601+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:45.529601+0000","last_peered":"2024-09-05T13:22:45.529601+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:45.529601+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719893+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:20:38.719893+0000","last_peered":"2024-09-05T13:20:38.719893+0000","last_clean":"2024-09-05T13:20:38.719893+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:20:38.719893+0000","last_undegraded":"2024-09-05T13:20:38.719893+0000","last_fullsized":"2024-09-05T13:20:38.719893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29819,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:22:45.502143+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:45.502143+0000","last_peered":"2024-09-05T13:22:45.502143+0000","last_clean":"2024-09-05T13:22:45.502143+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:45.502143+0000","last_undegraded":"2024-09-05T13:22:45.502143+0000","last_fullsized":"2024-09-05T13:22:45.502143+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":88,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883722+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:49.883722+0000","last_peered":"2024-09-05T13:22:49.883722+0000","last_clean":"2024-09-05T13:22:49.883722+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:49.883722+0000","last_undegraded":"2024-09-05T13:22:49.883722+0000","last_fullsized":"2024-09-05T13:22:49.883722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:45.483479+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:45.483479+0000","last_peered":"2024-09-05T13:22:45.483479+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:45.483479+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883834+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:49.883834+0000","last_peered":"2024-09-05T13:22:49.883834+0000","last_clean":"2024-09-05T13:22:49.883834+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:49.883834+0000","last_undegraded":"2024-09-05T13:22:49.883834+0000","last_fullsized":"2024-09-05T13:22:49.883834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-05T13:20:38.719934+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:20:38.719934+0000","last_peered":"2024-09-05T13:20:38.719934+0000","last_clean":"2024-09-05T13:20:38.719934+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:20:38.719934+0000","last_undegraded":"2024-09-05T13:20:38.719934+0000","last_fullsized":"2024-09-05T13:20:38.719934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21801,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:51.180387+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:22:51.180387+0000","last_peered":"2024-09-05T13:22:51.180387+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:51.180387+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.883885+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:49.883885+0000","last_peered":"2024-09-05T13:22:49.883885+0000","last_clean":"2024-09-05T13:22:49.883885+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:49.883885+0000","last_undegraded":"2024-09-05T13:22:49.883885+0000","last_fullsized":"2024-09-05T13:22:49.883885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-05T13:22:49.884038+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:49.884038+0000","last_peered":"2024-09-05T13:22:49.884038+0000","last_clean":"2024-09-05T13:22:49.884038+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:49.884038+0000","last_undegraded":"2024-09-05T13:22:49.884038+0000","last_fullsized":"2024-09-05T13:22:49.884038+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39890,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13608,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486076,"kb_used_data":143404,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885764,"statfs":{"total":96636764160,"available":96139022336,"internally_reserved":0,"allocated":146845696,"data_stored":146579766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562416,"kb_used_data":71260,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809424,"statfs":{"total":96636764160,"available":96060850176,"internally_reserved":0,"allocated":72970240,"data_stored":72694996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469460,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902380,"statfs":{"total":96636764160,"available":96156037120,"internally_reserved":0,"allocated":462848,"data_stored":138646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:55.693 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:56.005 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:56.006 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:56.018 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-05T13:22:55.751475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39890,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13609,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522152,"kb_used_data":215148,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281593368,"statfs":{"total":289910292480,"available":288351608832,"internally_reserved":0,"allocated":220311552,"data_stored":219425186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1338297350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1163288,"num_objects":-285,"num_object_clones":0,"num_object_copies":-570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001395"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21246,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896611+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896611+0000","last_peered":"2024-09-05T13:22:50.896611+0000","last_clean":"2024-09-05T13:22:50.896611+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896611+0000","last_undegraded":"2024-09-05T13:22:50.896611+0000","last_fullsized":"2024-09-05T13:22:50.896611+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21801,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:51.180387+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:22:51.180387+0000","last_peered":"2024-09-05T13:22:51.180387+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:51.180387+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39890,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13608,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562416,"kb_used_data":71260,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809424,"statfs":{"total":96636764160,"available":96060850176,"internally_reserved":0,"allocated":72970240,"data_stored":72694996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:56.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:22:56.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21057, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898893+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:50.898893+0000', 'last_peered': '2024-09-05T13:22:50.898893+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:50.898893+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:56.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:22:56.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21304, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897229+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:50.897229+0000', 'last_peered': '2024-09-05T13:22:50.897229+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:50.897229+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:56.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:22:56.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21239, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.895152+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:50.895152+0000', 'last_peered': '2024-09-05T13:22:50.895152+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:50.895152+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:56.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:22:56.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21059, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.896887+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:50.896887+0000', 'last_peered': '2024-09-05T13:22:50.896887+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:50.896887+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:56.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:22:56.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20716, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.893517+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:50.893517+0000', 'last_peered': '2024-09-05T13:22:50.893517+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:50.893517+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:22:56.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:22:56.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21029, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897067+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:50.897067+0000', 'last_peered': '2024-09-05T13:22:50.897067+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:50.897067+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:56.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:22:56.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21302, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898896+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:50.898896+0000', 'last_peered': '2024-09-05T13:22:50.898896+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:50.898896+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:56.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:22:56.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'21080", 'reported_seq': 21801, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:51.180387+0000', 'last_change': '2024-09-05T13:22:49.022873+0000', 'last_active': '2024-09-05T13:22:51.180387+0000', 'last_peered': '2024-09-05T13:22:51.180387+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:51.180387+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17210, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 285, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:56.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:22:56.024 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21343, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897170+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:50.897170+0000', 'last_peered': '2024-09-05T13:22:50.897170+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:50.897170+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:22:56.025 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:22:56.466 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:56.466 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39890/6 objects degraded (664833.333%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1558683648,"bytes_avail":288351608832,"bytes_total":289910292480,"degraded_objects":39890,"degraded_total":6,"degraded_ratio":6648.3333333333321,"recovering_objects_per_sec":149,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1198,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":35},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:22:56.479 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:22:56.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:22:56.796 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:22:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:22:56.810 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-05T13:22:55.751475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39890,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13609,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522152,"kb_used_data":215148,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281593368,"statfs":{"total":289910292480,"available":288351608832,"internally_reserved":0,"allocated":220311552,"data_stored":219425186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1338297350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1163288,"num_objects":-285,"num_object_clones":0,"num_object_copies":-570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001395"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21246,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896611+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896611+0000","last_peered":"2024-09-05T13:22:50.896611+0000","last_clean":"2024-09-05T13:22:50.896611+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896611+0000","last_undegraded":"2024-09-05T13:22:50.896611+0000","last_fullsized":"2024-09-05T13:22:50.896611+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21801,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:51.180387+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:22:51.180387+0000","last_peered":"2024-09-05T13:22:51.180387+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:51.180387+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39890,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13608,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562416,"kb_used_data":71260,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809424,"statfs":{"total":96636764160,"available":96060850176,"internally_reserved":0,"allocated":72970240,"data_stored":72694996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:22:59.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:00.110 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:00.111 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:00.123 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-05T13:22:59.752315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14108,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522164,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281593356,"statfs":{"total":289910292480,"available":288351596544,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1338297350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.869950"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":22799,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:56.178044+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:22:56.178044+0000","last_peered":"2024-09-05T13:22:56.178044+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:56.178044+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3784}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14107,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:00.124 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:00.425 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:00.437 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-05T13:22:59.752315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14108,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522164,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281593356,"statfs":{"total":289910292480,"available":288351596544,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1338297350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.869950"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":22799,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:56.178044+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:22:56.178044+0000","last_peered":"2024-09-05T13:22:56.178044+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:56.178044+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3784}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14107,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:00.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:00.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21057, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898893+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:50.898893+0000', 'last_peered': '2024-09-05T13:22:50.898893+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:50.898893+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:00.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:00.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21304, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897229+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:50.897229+0000', 'last_peered': '2024-09-05T13:22:50.897229+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:50.897229+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:00.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21239, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.895152+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:50.895152+0000', 'last_peered': '2024-09-05T13:22:50.895152+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:50.895152+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21059, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.896887+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:50.896887+0000', 'last_peered': '2024-09-05T13:22:50.896887+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:50.896887+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20716, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.893517+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:50.893517+0000', 'last_peered': '2024-09-05T13:22:50.893517+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:50.893517+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21029, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897067+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:50.897067+0000', 'last_peered': '2024-09-05T13:22:50.897067+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:50.897067+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21302, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898896+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:50.898896+0000', 'last_peered': '2024-09-05T13:22:50.898896+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:50.898896+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:23:00.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'21080", 'reported_seq': 22799, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:56.178044+0000', 'last_change': '2024-09-05T13:22:49.022873+0000', 'last_active': '2024-09-05T13:22:56.178044+0000', 'last_peered': '2024-09-05T13:22:56.178044+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:22:56.178044+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3784, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17210, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 784, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3784}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:00.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:00.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21343, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897170+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:50.897170+0000', 'last_peered': '2024-09-05T13:22:50.897170+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:50.897170+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:00.440 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:00.822 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:00.822 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39391/6 objects degraded (656516.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":707,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1558695936,"bytes_avail":288351596544,"bytes_total":289910292480,"degraded_objects":39391,"degraded_total":6,"degraded_ratio":6565.1666666666679,"recovering_objects_per_sec":79,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:00.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:00.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:01.158 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-05T13:22:59.752315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14108,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522164,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1306931,"kb_avail":281593356,"statfs":{"total":289910292480,"available":288351596544,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1338297350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.869950"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":22799,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:56.178044+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:22:56.178044+0000","last_peered":"2024-09-05T13:22:56.178044+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:22:56.178044+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3784}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14107,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473636,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898204,"statfs":{"total":96636764160,"available":96151760896,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:04.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:04.474 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:04.475 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:04.487 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-05T13:23:03.753096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14652,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509684,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281605836,"statfs":{"total":289910292480,"available":288364376064,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1325517830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1043,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002344"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":23887,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:01.181560+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:01.181560+0000","last_peered":"2024-09-05T13:23:01.181560+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:01.181560+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14651,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:04.489 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:04.789 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:04.789 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:04.803 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-05T13:23:03.753096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14652,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509684,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281605836,"statfs":{"total":289910292480,"available":288364376064,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1325517830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1043,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002344"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":23887,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:01.181560+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:01.181560+0000","last_peered":"2024-09-05T13:23:01.181560+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:01.181560+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14651,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:04.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21057, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898893+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:50.898893+0000', 'last_peered': '2024-09-05T13:22:50.898893+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:50.898893+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21304, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897229+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:50.897229+0000', 'last_peered': '2024-09-05T13:22:50.897229+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:50.897229+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21239, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.895152+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:50.895152+0000', 'last_peered': '2024-09-05T13:22:50.895152+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:50.895152+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21059, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.896887+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:50.896887+0000', 'last_peered': '2024-09-05T13:22:50.896887+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:50.896887+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20716, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.893517+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:50.893517+0000', 'last_peered': '2024-09-05T13:22:50.893517+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:50.893517+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21029, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897067+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:50.897067+0000', 'last_peered': '2024-09-05T13:22:50.897067+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:50.897067+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21302, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898896+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:50.898896+0000', 'last_peered': '2024-09-05T13:22:50.898896+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:50.898896+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:23:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'21080", 'reported_seq': 23887, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:01.181560+0000', 'last_change': '2024-09-05T13:22:49.022873+0000', 'last_active': '2024-09-05T13:23:01.181560+0000', 'last_peered': '2024-09-05T13:23:01.181560+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:23:01.181560+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3240, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17210, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1328, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3240}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21343, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897170+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:50.897170+0000', 'last_peered': '2024-09-05T13:22:50.897170+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:50.897170+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:04.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:05.219 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:05.219 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38847/6 objects degraded (647450.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":712,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1545916416,"bytes_avail":288364376064,"bytes_total":289910292480,"degraded_objects":38847,"degraded_total":6,"degraded_ratio":6474.5,"recovering_objects_per_sec":86,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1043,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:05.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:05.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:05.552 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:05.552 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:05.564 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-05T13:23:03.753096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14652,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509684,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281605836,"statfs":{"total":289910292480,"available":288364376064,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1325517830},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1043,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002344"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":23887,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:01.181560+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:01.181560+0000","last_peered":"2024-09-05T13:23:01.181560+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:01.181560+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14651,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461156,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910684,"statfs":{"total":96636764160,"available":96164540416,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:08.566 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:08.894 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:08.894 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:08.906 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-05T13:23:07.753728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15251,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513844,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281601676,"statfs":{"total":289910292480,"available":288360116224,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1329777670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002252"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":25085,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:06.186473+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:06.186473+0000","last_peered":"2024-09-05T13:23:06.186473+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:06.186473+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2641,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1927,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15250,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465316,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906524,"statfs":{"total":96636764160,"available":96160280576,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:08.908 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:09.233 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:09.234 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:09.246 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-05T13:23:07.753728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15251,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513844,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281601676,"statfs":{"total":289910292480,"available":288360116224,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1329777670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002252"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":25085,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:06.186473+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:06.186473+0000","last_peered":"2024-09-05T13:23:06.186473+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:06.186473+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2641,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1927,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15250,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465316,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906524,"statfs":{"total":96636764160,"available":96160280576,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:09.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:09.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21057, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898893+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:50.898893+0000', 'last_peered': '2024-09-05T13:22:50.898893+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:50.898893+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:09.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:09.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21304, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897229+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:50.897229+0000', 'last_peered': '2024-09-05T13:22:50.897229+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:50.897229+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21239, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.895152+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:50.895152+0000', 'last_peered': '2024-09-05T13:22:50.895152+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:50.895152+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21059, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.896887+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:50.896887+0000', 'last_peered': '2024-09-05T13:22:50.896887+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:50.896887+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20716, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.893517+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:50.893517+0000', 'last_peered': '2024-09-05T13:22:50.893517+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:50.893517+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21029, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897067+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:50.897067+0000', 'last_peered': '2024-09-05T13:22:50.897067+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:50.897067+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:09.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21302, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898896+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:50.898896+0000', 'last_peered': '2024-09-05T13:22:50.898896+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:50.898896+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:09.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:23:09.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'21080", 'reported_seq': 25085, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:06.186473+0000', 'last_change': '2024-09-05T13:22:49.022873+0000', 'last_active': '2024-09-05T13:23:06.186473+0000', 'last_peered': '2024-09-05T13:23:06.186473+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:23:06.186473+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2641, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17210, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1927, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2641}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:09.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:09.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21343, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897170+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:50.897170+0000', 'last_peered': '2024-09-05T13:22:50.897170+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:50.897170+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:09.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:09.650 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:09.651 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38248/6 objects degraded (637466.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":716,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1550176256,"bytes_avail":288360116224,"bytes_total":289910292480,"degraded_objects":38248,"degraded_total":6,"degraded_ratio":6374.6666666666679,"recovering_objects_per_sec":136,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:09.666 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:09.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:09.998 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:10.012 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-05T13:23:09.754074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15251,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513844,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281601676,"statfs":{"total":289910292480,"available":288360116224,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1329777670},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002129"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":25085,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:06.186473+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:06.186473+0000","last_peered":"2024-09-05T13:23:06.186473+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:06.186473+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2641,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1927,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15250,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465316,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906524,"statfs":{"total":96636764160,"available":96160280576,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:13.014 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:13.312 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:13.324 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-05T13:23:11.754423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15930,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536052,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281579468,"statfs":{"total":289910292480,"available":288337375232,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1352518662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1822,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002108"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":26443,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:11.187667+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:11.187667+0000","last_peered":"2024-09-05T13:23:11.187667+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:11.187667+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2606,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1962}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15929,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487524,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884316,"statfs":{"total":96636764160,"available":96137539584,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:13.325 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:13.645 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:13.646 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:13.659 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-05T13:23:11.754423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15930,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536052,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281579468,"statfs":{"total":289910292480,"available":288337375232,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1352518662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1822,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002108"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":26443,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:11.187667+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:11.187667+0000","last_peered":"2024-09-05T13:23:11.187667+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:11.187667+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2606,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1962}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15929,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487524,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884316,"statfs":{"total":96636764160,"available":96137539584,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:13.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21057, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898893+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:50.898893+0000', 'last_peered': '2024-09-05T13:22:50.898893+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:50.898893+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21304, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897229+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:50.897229+0000', 'last_peered': '2024-09-05T13:22:50.897229+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:50.897229+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21239, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.895152+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:50.895152+0000', 'last_peered': '2024-09-05T13:22:50.895152+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:50.895152+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21059, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.896887+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:50.896887+0000', 'last_peered': '2024-09-05T13:22:50.896887+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:50.896887+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:13.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20716, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.893517+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:50.893517+0000', 'last_peered': '2024-09-05T13:22:50.893517+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:50.893517+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:13.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:13.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21029, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897067+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:50.897067+0000', 'last_peered': '2024-09-05T13:22:50.897067+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:50.897067+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:13.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:13.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21302, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898896+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:50.898896+0000', 'last_peered': '2024-09-05T13:22:50.898896+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:50.898896+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:13.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:23:13.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'21080", 'reported_seq': 26443, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:11.187667+0000', 'last_change': '2024-09-05T13:22:49.022873+0000', 'last_active': '2024-09-05T13:23:11.187667+0000', 'last_peered': '2024-09-05T13:23:11.187667+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:23:11.187667+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1962, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17210, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2606, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1962}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:13.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:13.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21343, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897170+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:50.897170+0000', 'last_peered': '2024-09-05T13:22:50.897170+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:50.897170+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:13.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:14.044 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:14.044 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37569/6 objects degraded (626150.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":720,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1572917248,"bytes_avail":288337375232,"bytes_total":289910292480,"degraded_objects":37569,"degraded_total":6,"degraded_ratio":6261.5,"recovering_objects_per_sec":106,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1278,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:14.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:14.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:14.375 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:14.376 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:14.387 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-05T13:23:13.754771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15930,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536052,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281579468,"statfs":{"total":289910292480,"available":288337375232,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1352518662},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002093"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":26443,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:11.187667+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:11.187667+0000","last_peered":"2024-09-05T13:23:11.187667+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:11.187667+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2606,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1962}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15929,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487524,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884316,"statfs":{"total":96636764160,"available":96137539584,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:17.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:17.738 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:17.738 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:17.753 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-05T13:23:15.755186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15930,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527732,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281587788,"statfs":{"total":289910292480,"available":288345894912,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1343998982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002090"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":26443,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:11.187667+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:11.187667+0000","last_peered":"2024-09-05T13:23:11.187667+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:11.187667+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2606,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1962}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37569,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15929,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479204,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892636,"statfs":{"total":96636764160,"available":96146059264,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:17.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:18.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:18.064 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:18.077 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-05T13:23:17.755521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16823,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527732,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281587788,"statfs":{"total":289910292480,"available":288345894912,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1343998982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002122"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":28229,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:16.189809+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:16.189809+0000","last_peered":"2024-09-05T13:23:16.189809+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:16.189809+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3499,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16822,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479204,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892636,"statfs":{"total":96636764160,"available":96146059264,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:18.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21057, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898893+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:22:50.898893+0000', 'last_peered': '2024-09-05T13:22:50.898893+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:22:50.898893+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21304, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897229+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:22:50.897229+0000', 'last_peered': '2024-09-05T13:22:50.897229+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:22:50.897229+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21239, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.895152+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:50.895152+0000', 'last_peered': '2024-09-05T13:22:50.895152+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:50.895152+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21059, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.896887+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:22:50.896887+0000', 'last_peered': '2024-09-05T13:22:50.896887+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:22:50.896887+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20716, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.893517+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:50.893517+0000', 'last_peered': '2024-09-05T13:22:50.893517+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:50.893517+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:18.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:18.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21029, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897067+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:22:50.897067+0000', 'last_peered': '2024-09-05T13:22:50.897067+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:22:50.897067+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:18.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:18.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21302, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.898896+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:22:50.898896+0000', 'last_peered': '2024-09-05T13:22:50.898896+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:22:50.898896+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:18.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:23:18.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'21080", 'reported_seq': 28229, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:16.189809+0000', 'last_change': '2024-09-05T13:22:49.022873+0000', 'last_active': '2024-09-05T13:23:16.189809+0000', 'last_peered': '2024-09-05T13:23:16.189809+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:23:16.189809+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1069, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17210, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3499, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1069}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:18.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:18.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21343, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.897170+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:22:50.897170+0000', 'last_peered': '2024-09-05T13:22:50.897170+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:22:50.897170+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:18.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:18.498 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:18.498 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36676/6 objects degraded (611266.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1564397568,"bytes_avail":288345894912,"bytes_total":289910292480,"degraded_objects":36676,"degraded_total":6,"degraded_ratio":6112.6666666666679,"recovering_objects_per_sec":180,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2171,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:18.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:18.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:18.807 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:18.808 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:18.820 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-05T13:23:17.755521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16823,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527732,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281587788,"statfs":{"total":289910292480,"available":288345894912,"internally_reserved":0,"allocated":220319744,"data_stored":219428120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1343998982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002122"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898893+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:22:50.898893+0000","last_peered":"2024-09-05T13:22:50.898893+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:22:50.898893+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897054+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:22:50.897054+0000","last_peered":"2024-09-05T13:22:50.897054+0000","last_clean":"2024-09-05T13:22:50.897054+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:22:50.897054+0000","last_undegraded":"2024-09-05T13:22:50.897054+0000","last_fullsized":"2024-09-05T13:22:50.897054+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21304,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897229+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:22:50.897229+0000","last_peered":"2024-09-05T13:22:50.897229+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:22:50.897229+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21247,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896656+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:22:50.896656+0000","last_peered":"2024-09-05T13:22:50.896656+0000","last_clean":"2024-09-05T13:22:50.896656+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:22:50.896656+0000","last_undegraded":"2024-09-05T13:22:50.896656+0000","last_fullsized":"2024-09-05T13:22:50.896656+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897040+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897040+0000","last_peered":"2024-09-05T13:22:50.897040+0000","last_clean":"2024-09-05T13:22:50.897040+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897040+0000","last_undegraded":"2024-09-05T13:22:50.897040+0000","last_fullsized":"2024-09-05T13:22:50.897040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.896887+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:22:50.896887+0000","last_peered":"2024-09-05T13:22:50.896887+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:22:50.896887+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":115,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.896786+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:22:50.896786+0000","last_peered":"2024-09-05T13:22:50.896786+0000","last_clean":"2024-09-05T13:22:50.896786+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:22:50.896786+0000","last_undegraded":"2024-09-05T13:22:50.896786+0000","last_fullsized":"2024-09-05T13:22:50.896786+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21029,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897067+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:22:50.897067+0000","last_peered":"2024-09-05T13:22:50.897067+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:22:50.897067+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21302,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.898896+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:22:50.898896+0000","last_peered":"2024-09-05T13:22:50.898896+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:22:50.898896+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":28229,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:16.189809+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:16.189809+0000","last_peered":"2024-09-05T13:23:16.189809+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:16.189809+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3499,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21343,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.897170+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:22:50.897170+0000","last_peered":"2024-09-05T13:22:50.897170+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:22:50.897170+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16822,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219082752,"data_stored":219017718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562428,"kb_used_data":71268,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809412,"statfs":{"total":96636764160,"available":96060837888,"internally_reserved":0,"allocated":72978432,"data_stored":72697930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479204,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892636,"statfs":{"total":96636764160,"available":96146059264,"internally_reserved":0,"allocated":479232,"data_stored":144535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:21.822 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:22.210 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-05T13:23:22.077547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16823,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540208,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575312,"statfs":{"total":289910292480,"available":288333119488,"internally_reserved":0,"allocated":220319744,"data_stored":219430084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":893,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.322775"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21060,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.080474+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:21.080474+0000","last_peered":"2024-09-05T13:23:21.080474+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:21.080474+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21030,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.078298+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:21.078298+0000","last_peered":"2024-09-05T13:23:21.078298+0000","last_clean":"2024-09-05T13:23:21.078298+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:21.078298+0000","last_undegraded":"2024-09-05T13:23:21.078298+0000","last_fullsized":"2024-09-05T13:23:21.078298+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.077757+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:21.077757+0000","last_peered":"2024-09-05T13:23:21.077757+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:21.077757+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21249,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077417+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:21.077417+0000","last_peered":"2024-09-05T13:23:21.077417+0000","last_clean":"2024-09-05T13:23:21.077417+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:21.077417+0000","last_undegraded":"2024-09-05T13:23:21.077417+0000","last_fullsized":"2024-09-05T13:23:21.077417+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897086+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897086+0000","last_peered":"2024-09-05T13:22:50.897086+0000","last_clean":"2024-09-05T13:22:50.897086+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897086+0000","last_undegraded":"2024-09-05T13:22:50.897086+0000","last_fullsized":"2024-09-05T13:22:50.897086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21061,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078170+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:21.078170+0000","last_peered":"2024-09-05T13:23:21.078170+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:21.078170+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":117,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077672+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:21.077672+0000","last_peered":"2024-09-05T13:23:21.077672+0000","last_clean":"2024-09-05T13:23:21.077672+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:21.077672+0000","last_undegraded":"2024-09-05T13:23:21.077672+0000","last_fullsized":"2024-09-05T13:23:21.077672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078887+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:21.078887+0000","last_peered":"2024-09-05T13:23:21.078887+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:21.078887+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21304,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078661+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:21.078661+0000","last_peered":"2024-09-05T13:23:21.078661+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:21.078661+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":28229,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:16.189809+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:16.189809+0000","last_peered":"2024-09-05T13:23:16.189809+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:16.189809+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3499,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21345,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078378+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:21.078378+0000","last_peered":"2024-09-05T13:23:21.078378+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:21.078378+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16822,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562432,"kb_used_data":71276,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809408,"statfs":{"total":96636764160,"available":96060833792,"internally_reserved":0,"allocated":72986624,"data_stored":72700860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491676,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880164,"statfs":{"total":96636764160,"available":96133287936,"internally_reserved":0,"allocated":471040,"data_stored":143569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:22.212 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:22.576 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-05T13:23:22.077547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16823,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540208,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575312,"statfs":{"total":289910292480,"available":288333119488,"internally_reserved":0,"allocated":220319744,"data_stored":219430084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":893,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.322775"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21060,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.080474+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:21.080474+0000","last_peered":"2024-09-05T13:23:21.080474+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:21.080474+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21030,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.078298+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:21.078298+0000","last_peered":"2024-09-05T13:23:21.078298+0000","last_clean":"2024-09-05T13:23:21.078298+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:21.078298+0000","last_undegraded":"2024-09-05T13:23:21.078298+0000","last_fullsized":"2024-09-05T13:23:21.078298+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.077757+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:21.077757+0000","last_peered":"2024-09-05T13:23:21.077757+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:21.077757+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21249,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077417+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:21.077417+0000","last_peered":"2024-09-05T13:23:21.077417+0000","last_clean":"2024-09-05T13:23:21.077417+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:21.077417+0000","last_undegraded":"2024-09-05T13:23:21.077417+0000","last_fullsized":"2024-09-05T13:23:21.077417+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897086+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897086+0000","last_peered":"2024-09-05T13:22:50.897086+0000","last_clean":"2024-09-05T13:22:50.897086+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897086+0000","last_undegraded":"2024-09-05T13:22:50.897086+0000","last_fullsized":"2024-09-05T13:22:50.897086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21061,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078170+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:21.078170+0000","last_peered":"2024-09-05T13:23:21.078170+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:21.078170+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":117,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077672+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:21.077672+0000","last_peered":"2024-09-05T13:23:21.077672+0000","last_clean":"2024-09-05T13:23:21.077672+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:21.077672+0000","last_undegraded":"2024-09-05T13:23:21.077672+0000","last_fullsized":"2024-09-05T13:23:21.077672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078887+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:21.078887+0000","last_peered":"2024-09-05T13:23:21.078887+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:21.078887+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21304,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078661+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:21.078661+0000","last_peered":"2024-09-05T13:23:21.078661+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:21.078661+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":28229,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:16.189809+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:16.189809+0000","last_peered":"2024-09-05T13:23:16.189809+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:16.189809+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3499,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21345,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078378+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:21.078378+0000","last_peered":"2024-09-05T13:23:21.078378+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:21.078378+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16822,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562432,"kb_used_data":71276,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809408,"statfs":{"total":96636764160,"available":96060833792,"internally_reserved":0,"allocated":72986624,"data_stored":72700860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491676,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880164,"statfs":{"total":96636764160,"available":96133287936,"internally_reserved":0,"allocated":471040,"data_stored":143569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21060, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.080474+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:23:21.080474+0000', 'last_peered': '2024-09-05T13:23:21.080474+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:21.080474+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21306, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.077757+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:21.077757+0000', 'last_peered': '2024-09-05T13:23:21.077757+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:21.077757+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21239, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.895152+0000', 'last_change': '2024-09-05T13:13:18.493619+0000', 'last_active': '2024-09-05T13:22:50.895152+0000', 'last_peered': '2024-09-05T13:22:50.895152+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:22:50.895152+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21061, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.078170+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:21.078170+0000', 'last_peered': '2024-09-05T13:23:21.078170+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:21.078170+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:22.578 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20716, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:22:50.893517+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:22:50.893517+0000', 'last_peered': '2024-09-05T13:22:50.893517+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:22:50.893517+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:22.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:22.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21031, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.078887+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:21.078887+0000', 'last_peered': '2024-09-05T13:23:21.078887+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:21.078887+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:22.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:22.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21304, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.078661+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:21.078661+0000', 'last_peered': '2024-09-05T13:23:21.078661+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:21.078661+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:22.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-05T13:23:22.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'21080", 'reported_seq': 28229, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:16.189809+0000', 'last_change': '2024-09-05T13:22:49.022873+0000', 'last_active': '2024-09-05T13:23:16.189809+0000', 'last_peered': '2024-09-05T13:23:16.189809+0000', 'last_clean': '2024-09-05T13:13:15.888158+0000', 'last_became_active': '2024-09-05T13:13:18.465726+0000', 'last_became_peered': '2024-09-05T13:13:18.465726+0000', 'last_unstale': '2024-09-05T13:23:16.189809+0000', 'last_undegraded': '2024-09-05T13:13:17.988029+0000', 'last_fullsized': '2024-09-05T13:13:17.901879+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3904", 'last_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:05.462886+0000', 'objects_scrubbed': 3866, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1394, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1069, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17210, 'num_write_kb': 26688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3499, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1069}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:22.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:22.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21345, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.078378+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:21.078378+0000', 'last_peered': '2024-09-05T13:23:21.078378+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:21.078378+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:22.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:22.959 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:22.960 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36676/6 objects degraded (611266.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":729,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1577172992,"bytes_avail":288333119488,"bytes_total":289910292480,"degraded_objects":36676,"degraded_total":6,"degraded_ratio":6112.6666666666679,"recovering_objects_per_sec":89,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":893,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:22.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:22.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:23.317 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:23.318 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:23.330 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-05T13:23:22.077547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16823,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540208,"kb_used_data":215156,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575312,"statfs":{"total":289910292480,"available":288333119488,"internally_reserved":0,"allocated":220319744,"data_stored":219430084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":893,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.322775"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21060,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.080474+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:21.080474+0000","last_peered":"2024-09-05T13:23:21.080474+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:21.080474+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20869,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894074+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:22:50.894074+0000","last_peered":"2024-09-05T13:22:50.894074+0000","last_clean":"2024-09-05T13:22:50.894074+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:22:50.894074+0000","last_undegraded":"2024-09-05T13:22:50.894074+0000","last_fullsized":"2024-09-05T13:22:50.894074+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.969397+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:22:50.969397+0000","last_peered":"2024-09-05T13:22:50.969397+0000","last_clean":"2024-09-05T13:22:50.969397+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:22:50.969397+0000","last_undegraded":"2024-09-05T13:22:50.969397+0000","last_fullsized":"2024-09-05T13:22:50.969397+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.895517+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:22:50.895517+0000","last_peered":"2024-09-05T13:22:50.895517+0000","last_clean":"2024-09-05T13:22:50.895517+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:22:50.895517+0000","last_undegraded":"2024-09-05T13:22:50.895517+0000","last_fullsized":"2024-09-05T13:22:50.895517+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21030,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.078298+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:21.078298+0000","last_peered":"2024-09-05T13:23:21.078298+0000","last_clean":"2024-09-05T13:23:21.078298+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:21.078298+0000","last_undegraded":"2024-09-05T13:23:21.078298+0000","last_fullsized":"2024-09-05T13:23:21.078298+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.077757+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:21.077757+0000","last_peered":"2024-09-05T13:23:21.077757+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:21.077757+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897261+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:22:50.897261+0000","last_peered":"2024-09-05T13:22:50.897261+0000","last_clean":"2024-09-05T13:22:50.897261+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:22:50.897261+0000","last_undegraded":"2024-09-05T13:22:50.897261+0000","last_fullsized":"2024-09-05T13:22:50.897261+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21249,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077417+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:21.077417+0000","last_peered":"2024-09-05T13:23:21.077417+0000","last_clean":"2024-09-05T13:23:21.077417+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:21.077417+0000","last_undegraded":"2024-09-05T13:23:21.077417+0000","last_fullsized":"2024-09-05T13:23:21.077417+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897086+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:22:50.897086+0000","last_peered":"2024-09-05T13:22:50.897086+0000","last_clean":"2024-09-05T13:22:50.897086+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:22:50.897086+0000","last_undegraded":"2024-09-05T13:22:50.897086+0000","last_fullsized":"2024-09-05T13:22:50.897086+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21239,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.895152+0000","last_change":"2024-09-05T13:13:18.493619+0000","last_active":"2024-09-05T13:22:50.895152+0000","last_peered":"2024-09-05T13:22:50.895152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:22:50.895152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893633+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:22:50.893633+0000","last_peered":"2024-09-05T13:22:50.893633+0000","last_clean":"2024-09-05T13:22:50.893633+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:22:50.893633+0000","last_undegraded":"2024-09-05T13:22:50.893633+0000","last_fullsized":"2024-09-05T13:22:50.893633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21061,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078170+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:21.078170+0000","last_peered":"2024-09-05T13:23:21.078170+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:21.078170+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":117,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077672+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:21.077672+0000","last_peered":"2024-09-05T13:23:21.077672+0000","last_clean":"2024-09-05T13:23:21.077672+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:21.077672+0000","last_undegraded":"2024-09-05T13:23:21.077672+0000","last_fullsized":"2024-09-05T13:23:21.077672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.893713+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:22:50.893713+0000","last_peered":"2024-09-05T13:22:50.893713+0000","last_clean":"2024-09-05T13:22:50.893713+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:22:50.893713+0000","last_undegraded":"2024-09-05T13:22:50.893713+0000","last_fullsized":"2024-09-05T13:22:50.893713+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897547+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:22:50.897547+0000","last_peered":"2024-09-05T13:22:50.897547+0000","last_clean":"2024-09-05T13:22:50.897547+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:22:50.897547+0000","last_undegraded":"2024-09-05T13:22:50.897547+0000","last_fullsized":"2024-09-05T13:22:50.897547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20716,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:22:50.893517+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:22:50.893517+0000","last_peered":"2024-09-05T13:22:50.893517+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:22:50.893517+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078887+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:21.078887+0000","last_peered":"2024-09-05T13:23:21.078887+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:21.078887+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897710+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:22:50.897710+0000","last_peered":"2024-09-05T13:22:50.897710+0000","last_clean":"2024-09-05T13:22:50.897710+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:22:50.897710+0000","last_undegraded":"2024-09-05T13:22:50.897710+0000","last_fullsized":"2024-09-05T13:22:50.897710+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21304,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078661+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:21.078661+0000","last_peered":"2024-09-05T13:23:21.078661+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:21.078661+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.894373+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:22:50.894373+0000","last_peered":"2024-09-05T13:22:50.894373+0000","last_clean":"2024-09-05T13:22:50.894373+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:22:50.894373+0000","last_undegraded":"2024-09-05T13:22:50.894373+0000","last_fullsized":"2024-09-05T13:22:50.894373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":28229,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:16.189809+0000","last_change":"2024-09-05T13:22:49.022873+0000","last_active":"2024-09-05T13:23:16.189809+0000","last_peered":"2024-09-05T13:23:16.189809+0000","last_clean":"2024-09-05T13:13:15.888158+0000","last_became_active":"2024-09-05T13:13:18.465726+0000","last_became_peered":"2024-09-05T13:13:18.465726+0000","last_unstale":"2024-09-05T13:23:16.189809+0000","last_undegraded":"2024-09-05T13:13:17.988029+0000","last_fullsized":"2024-09-05T13:13:17.901879+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3499,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897652+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:22:50.897652+0000","last_peered":"2024-09-05T13:22:50.897652+0000","last_clean":"2024-09-05T13:22:50.897652+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:22:50.897652+0000","last_undegraded":"2024-09-05T13:22:50.897652+0000","last_fullsized":"2024-09-05T13:22:50.897652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-05T13:22:50.897799+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:22:50.897799+0000","last_peered":"2024-09-05T13:22:50.897799+0000","last_clean":"2024-09-05T13:22:50.897799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:22:50.897799+0000","last_undegraded":"2024-09-05T13:22:50.897799+0000","last_fullsized":"2024-09-05T13:22:50.897799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21345,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078378+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:21.078378+0000","last_peered":"2024-09-05T13:23:21.078378+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:21.078378+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16822,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486100,"kb_used_data":143420,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885740,"statfs":{"total":96636764160,"available":96138997760,"internally_reserved":0,"allocated":146862080,"data_stored":146585655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562432,"kb_used_data":71276,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809408,"statfs":{"total":96636764160,"available":96060833792,"internally_reserved":0,"allocated":72986624,"data_stored":72700860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491676,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880164,"statfs":{"total":96636764160,"available":96133287936,"internally_reserved":0,"allocated":471040,"data_stored":143569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:26.332 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:26.733 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:26.733 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:26.746 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-05T13:23:25.756866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18505,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527756,"kb_used_data":215188,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281587764,"statfs":{"total":289910292480,"available":288345870336,"internally_reserved":0,"allocated":220352512,"data_stored":219441762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1343998982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001344"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21060,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.080474+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:21.080474+0000","last_peered":"2024-09-05T13:23:21.080474+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:21.080474+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20873,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087258+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087258+0000","last_peered":"2024-09-05T13:23:22.087258+0000","last_clean":"2024-09-05T13:23:22.087258+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087258+0000","last_undegraded":"2024-09-05T13:23:22.087258+0000","last_fullsized":"2024-09-05T13:23:22.087258+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086924+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086924+0000","last_peered":"2024-09-05T13:23:22.086924+0000","last_clean":"2024-09-05T13:23:22.086924+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086924+0000","last_undegraded":"2024-09-05T13:23:22.086924+0000","last_fullsized":"2024-09-05T13:23:22.086924+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21030,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.078298+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:21.078298+0000","last_peered":"2024-09-05T13:23:21.078298+0000","last_clean":"2024-09-05T13:23:21.078298+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:21.078298+0000","last_undegraded":"2024-09-05T13:23:21.078298+0000","last_fullsized":"2024-09-05T13:23:21.078298+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.077757+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:21.077757+0000","last_peered":"2024-09-05T13:23:21.077757+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:21.077757+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21249,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077417+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:21.077417+0000","last_peered":"2024-09-05T13:23:21.077417+0000","last_clean":"2024-09-05T13:23:21.077417+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:21.077417+0000","last_undegraded":"2024-09-05T13:23:21.077417+0000","last_fullsized":"2024-09-05T13:23:21.077417+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":22470,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:24.122219+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:24.122219+0000","last_peered":"2024-09-05T13:23:24.122219+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:24.122219+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":615,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21061,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078170+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:21.078170+0000","last_peered":"2024-09-05T13:23:21.078170+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:21.078170+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":117,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077672+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:21.077672+0000","last_peered":"2024-09-05T13:23:21.077672+0000","last_clean":"2024-09-05T13:23:21.077672+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:21.077672+0000","last_undegraded":"2024-09-05T13:23:21.077672+0000","last_fullsized":"2024-09-05T13:23:21.077672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078887+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:21.078887+0000","last_peered":"2024-09-05T13:23:21.078887+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:21.078887+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21304,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078661+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:21.078661+0000","last_peered":"2024-09-05T13:23:21.078661+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:21.078661+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21345,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078378+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:21.078378+0000","last_peered":"2024-09-05T13:23:21.078378+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:21.078378+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18504,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562432,"kb_used_data":71276,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809408,"statfs":{"total":96636764160,"available":96060833792,"internally_reserved":0,"allocated":72986624,"data_stored":72700860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:26.747 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:27.084 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:27.084 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-05T13:23:25.756866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18505,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527756,"kb_used_data":215188,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281587764,"statfs":{"total":289910292480,"available":288345870336,"internally_reserved":0,"allocated":220352512,"data_stored":219441762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1343998982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001344"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21060,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.080474+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:21.080474+0000","last_peered":"2024-09-05T13:23:21.080474+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:21.080474+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20873,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087258+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087258+0000","last_peered":"2024-09-05T13:23:22.087258+0000","last_clean":"2024-09-05T13:23:22.087258+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087258+0000","last_undegraded":"2024-09-05T13:23:22.087258+0000","last_fullsized":"2024-09-05T13:23:22.087258+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086924+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086924+0000","last_peered":"2024-09-05T13:23:22.086924+0000","last_clean":"2024-09-05T13:23:22.086924+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086924+0000","last_undegraded":"2024-09-05T13:23:22.086924+0000","last_fullsized":"2024-09-05T13:23:22.086924+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21030,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.078298+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:21.078298+0000","last_peered":"2024-09-05T13:23:21.078298+0000","last_clean":"2024-09-05T13:23:21.078298+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:21.078298+0000","last_undegraded":"2024-09-05T13:23:21.078298+0000","last_fullsized":"2024-09-05T13:23:21.078298+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21306,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.077757+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:21.077757+0000","last_peered":"2024-09-05T13:23:21.077757+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:21.077757+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21249,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077417+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:21.077417+0000","last_peered":"2024-09-05T13:23:21.077417+0000","last_clean":"2024-09-05T13:23:21.077417+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:21.077417+0000","last_undegraded":"2024-09-05T13:23:21.077417+0000","last_fullsized":"2024-09-05T13:23:21.077417+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":22470,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:24.122219+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:24.122219+0000","last_peered":"2024-09-05T13:23:24.122219+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:24.122219+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":615,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21061,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078170+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:21.078170+0000","last_peered":"2024-09-05T13:23:21.078170+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:21.078170+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":117,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-05T13:23:21.077672+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:21.077672+0000","last_peered":"2024-09-05T13:23:21.077672+0000","last_clean":"2024-09-05T13:23:21.077672+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:21.077672+0000","last_undegraded":"2024-09-05T13:23:21.077672+0000","last_fullsized":"2024-09-05T13:23:21.077672+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078887+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:21.078887+0000","last_peered":"2024-09-05T13:23:21.078887+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:21.078887+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21304,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078661+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:21.078661+0000","last_peered":"2024-09-05T13:23:21.078661+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:21.078661+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21345,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:21.078378+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:21.078378+0000","last_peered":"2024-09-05T13:23:21.078378+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:21.078378+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18504,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562432,"kb_used_data":71276,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809408,"statfs":{"total":96636764160,"available":96060833792,"internally_reserved":0,"allocated":72986624,"data_stored":72700860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21060, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.080474+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:23:21.080474+0000', 'last_peered': '2024-09-05T13:23:21.080474+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:21.080474+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21306, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.077757+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:21.077757+0000', 'last_peered': '2024-09-05T13:23:21.077757+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:21.077757+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 22470, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:24.122219+0000', 'last_change': '2024-09-05T13:23:20.386406+0000', 'last_active': '2024-09-05T13:23:24.122219+0000', 'last_peered': '2024-09-05T13:23:24.122219+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:23:24.122219+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3840, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 615, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21061, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.078170+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:21.078170+0000', 'last_peered': '2024-09-05T13:23:21.078170+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:21.078170+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:27.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20720, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.086353+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:22.086353+0000', 'last_peered': '2024-09-05T13:23:22.086353+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:22.086353+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:27.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:27.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21031, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.078887+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:21.078887+0000', 'last_peered': '2024-09-05T13:23:21.078887+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:21.078887+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:27.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:27.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21304, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.078661+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:21.078661+0000', 'last_peered': '2024-09-05T13:23:21.078661+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:21.078661+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:27.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:27.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21345, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:21.078378+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:21.078378+0000', 'last_peered': '2024-09-05T13:23:21.078378+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:21.078378+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:27.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:27.546 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:27.546 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34994/6 objects degraded (583233.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":734,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1564422144,"bytes_avail":288345870336,"bytes_total":289910292480,"degraded_objects":34994,"degraded_total":6,"degraded_ratio":5832.3333333333321,"recovering_objects_per_sec":210,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:27.563 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:27.563 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:27.896 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:27.897 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:27.910 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-05T13:23:27.757205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18505,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527764,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281587756,"statfs":{"total":289910292480,"available":288345862144,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1343998982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001358"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20873,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087258+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087258+0000","last_peered":"2024-09-05T13:23:22.087258+0000","last_clean":"2024-09-05T13:23:22.087258+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087258+0000","last_undegraded":"2024-09-05T13:23:22.087258+0000","last_fullsized":"2024-09-05T13:23:22.087258+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086924+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086924+0000","last_peered":"2024-09-05T13:23:22.086924+0000","last_clean":"2024-09-05T13:23:22.086924+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086924+0000","last_undegraded":"2024-09-05T13:23:22.086924+0000","last_fullsized":"2024-09-05T13:23:22.086924+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":22470,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:24.122219+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:24.122219+0000","last_peered":"2024-09-05T13:23:24.122219+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:24.122219+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3840,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":615,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3840}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18504,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:30.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:31.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:31.282 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:31.298 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-05T13:23:29.757643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19543,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527764,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281587756,"statfs":{"total":289910292480,"available":288345862144,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1343998982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2720,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.681088"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20873,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087258+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087258+0000","last_peered":"2024-09-05T13:23:22.087258+0000","last_clean":"2024-09-05T13:23:22.087258+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087258+0000","last_undegraded":"2024-09-05T13:23:22.087258+0000","last_fullsized":"2024-09-05T13:23:22.087258+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086924+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086924+0000","last_peered":"2024-09-05T13:23:22.086924+0000","last_clean":"2024-09-05T13:23:22.086924+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086924+0000","last_undegraded":"2024-09-05T13:23:22.086924+0000","last_fullsized":"2024-09-05T13:23:22.086924+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":24546,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:29.124211+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:29.124211+0000","last_peered":"2024-09-05T13:23:29.124211+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:29.124211+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1653,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2802}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19542,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:31.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:31.684 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:31.698 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-05T13:23:29.757643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19543,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527764,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281587756,"statfs":{"total":289910292480,"available":288345862144,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1343998982},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2720,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.681088"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20873,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087258+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087258+0000","last_peered":"2024-09-05T13:23:22.087258+0000","last_clean":"2024-09-05T13:23:22.087258+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087258+0000","last_undegraded":"2024-09-05T13:23:22.087258+0000","last_fullsized":"2024-09-05T13:23:22.087258+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086924+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086924+0000","last_peered":"2024-09-05T13:23:22.086924+0000","last_clean":"2024-09-05T13:23:22.086924+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086924+0000","last_undegraded":"2024-09-05T13:23:22.086924+0000","last_fullsized":"2024-09-05T13:23:22.086924+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":24546,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:29.124211+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:29.124211+0000","last_peered":"2024-09-05T13:23:29.124211+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:29.124211+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1653,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2802}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19542,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479212,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892628,"statfs":{"total":96636764160,"available":96146051072,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:31.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:31.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21062, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082564+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:23:22.082564+0000', 'last_peered': '2024-09-05T13:23:22.082564+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:22.082564+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:31.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:31.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21308, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080501+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:22.080501+0000', 'last_peered': '2024-09-05T13:23:22.080501+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:22.080501+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:31.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:31.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 24546, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:29.124211+0000', 'last_change': '2024-09-05T13:23:20.386406+0000', 'last_active': '2024-09-05T13:23:29.124211+0000', 'last_peered': '2024-09-05T13:23:29.124211+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:23:29.124211+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2802, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1653, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2802}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:31.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:31.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21063, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080709+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:22.080709+0000', 'last_peered': '2024-09-05T13:23:22.080709+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:22.080709+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:31.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:31.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20720, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.086353+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:22.086353+0000', 'last_peered': '2024-09-05T13:23:22.086353+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:22.086353+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:31.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:31.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21033, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082219+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:22.082219+0000', 'last_peered': '2024-09-05T13:23:22.082219+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:22.082219+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:31.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:31.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21306, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082210+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:22.082210+0000', 'last_peered': '2024-09-05T13:23:22.082210+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:22.082210+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:31.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:31.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21347, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080380+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:22.080380+0000', 'last_peered': '2024-09-05T13:23:22.080380+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:22.080380+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:31.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:32.126 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:32.126 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33956/6 objects degraded (565933.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":738,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1564430336,"bytes_avail":288345862144,"bytes_total":289910292480,"degraded_objects":33956,"degraded_total":6,"degraded_ratio":5659.3333333333321,"recovering_objects_per_sec":313,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2720,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:32.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:32.140 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:32.538 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:32.551 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-05T13:23:31.758067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19543,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540244,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575276,"statfs":{"total":289910292480,"available":288333082624,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2720,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001897"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20873,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087258+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087258+0000","last_peered":"2024-09-05T13:23:22.087258+0000","last_clean":"2024-09-05T13:23:22.087258+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087258+0000","last_undegraded":"2024-09-05T13:23:22.087258+0000","last_fullsized":"2024-09-05T13:23:22.087258+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086924+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086924+0000","last_peered":"2024-09-05T13:23:22.086924+0000","last_clean":"2024-09-05T13:23:22.086924+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086924+0000","last_undegraded":"2024-09-05T13:23:22.086924+0000","last_fullsized":"2024-09-05T13:23:22.086924+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":24546,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:29.124211+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:29.124211+0000","last_peered":"2024-09-05T13:23:29.124211+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:29.124211+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2802,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1653,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2802}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33956,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19542,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491692,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880148,"statfs":{"total":96636764160,"available":96133271552,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:35.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:35.887 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:35.900 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-05T13:23:35.758794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20792,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531924,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583596,"statfs":{"total":289910292480,"available":288341602304,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002204"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20874,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087299+0000","last_peered":"2024-09-05T13:23:22.087299+0000","last_clean":"2024-09-05T13:23:22.087299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087299+0000","last_undegraded":"2024-09-05T13:23:22.087299+0000","last_fullsized":"2024-09-05T13:23:22.087299+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21111,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086964+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086964+0000","last_peered":"2024-09-05T13:23:22.086964+0000","last_clean":"2024-09-05T13:23:22.086964+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086964+0000","last_undegraded":"2024-09-05T13:23:22.086964+0000","last_fullsized":"2024-09-05T13:23:22.086964+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":27044,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:34.124661+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:34.124661+0000","last_peered":"2024-09-05T13:23:34.124661+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:34.124661+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1553}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20791,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483372,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888468,"statfs":{"total":96636764160,"available":96141791232,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:35.901 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:36.244 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:36.258 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-05T13:23:35.758794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20792,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531924,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583596,"statfs":{"total":289910292480,"available":288341602304,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002204"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20874,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087299+0000","last_peered":"2024-09-05T13:23:22.087299+0000","last_clean":"2024-09-05T13:23:22.087299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087299+0000","last_undegraded":"2024-09-05T13:23:22.087299+0000","last_fullsized":"2024-09-05T13:23:22.087299+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21111,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086964+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086964+0000","last_peered":"2024-09-05T13:23:22.086964+0000","last_clean":"2024-09-05T13:23:22.086964+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086964+0000","last_undegraded":"2024-09-05T13:23:22.086964+0000","last_fullsized":"2024-09-05T13:23:22.086964+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":27044,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:34.124661+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:34.124661+0000","last_peered":"2024-09-05T13:23:34.124661+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:34.124661+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1553}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20791,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483372,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888468,"statfs":{"total":96636764160,"available":96141791232,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:36.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:36.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21062, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082564+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:23:22.082564+0000', 'last_peered': '2024-09-05T13:23:22.082564+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:22.082564+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:36.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:36.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21308, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080501+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:22.080501+0000', 'last_peered': '2024-09-05T13:23:22.080501+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:22.080501+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:36.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:36.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 27044, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:34.124661+0000', 'last_change': '2024-09-05T13:23:20.386406+0000', 'last_active': '2024-09-05T13:23:34.124661+0000', 'last_peered': '2024-09-05T13:23:34.124661+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:23:34.124661+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1553, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2902, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1553}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:36.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21063, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080709+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:22.080709+0000', 'last_peered': '2024-09-05T13:23:22.080709+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:22.080709+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20720, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.086353+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:22.086353+0000', 'last_peered': '2024-09-05T13:23:22.086353+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:22.086353+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21033, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082219+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:22.082219+0000', 'last_peered': '2024-09-05T13:23:22.082219+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:22.082219+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21306, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082210+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:22.082210+0000', 'last_peered': '2024-09-05T13:23:22.082210+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:22.082210+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:36.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21347, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080380+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:22.080380+0000', 'last_peered': '2024-09-05T13:23:22.080380+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:22.080380+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:36.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:36.658 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:36.659 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32707/6 objects degraded (545116.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":743,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1568690176,"bytes_avail":288341602304,"bytes_total":289910292480,"degraded_objects":32707,"degraded_total":6,"degraded_ratio":5451.1666666666679,"recovering_objects_per_sec":330,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":3969,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:36.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:36.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:36.990 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:36.990 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:37.004 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-05T13:23:35.758794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20792,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531924,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583596,"statfs":{"total":289910292480,"available":288341602304,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002204"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20874,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087299+0000","last_peered":"2024-09-05T13:23:22.087299+0000","last_clean":"2024-09-05T13:23:22.087299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087299+0000","last_undegraded":"2024-09-05T13:23:22.087299+0000","last_fullsized":"2024-09-05T13:23:22.087299+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21111,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086964+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086964+0000","last_peered":"2024-09-05T13:23:22.086964+0000","last_clean":"2024-09-05T13:23:22.086964+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086964+0000","last_undegraded":"2024-09-05T13:23:22.086964+0000","last_fullsized":"2024-09-05T13:23:22.086964+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":27044,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:34.124661+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:34.124661+0000","last_peered":"2024-09-05T13:23:34.124661+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:34.124661+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1553}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20791,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483372,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888468,"statfs":{"total":96636764160,"available":96141791232,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:40.005 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:40.355 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:40.355 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:40.368 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-05T13:23:40.201898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22233,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531924,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583596,"statfs":{"total":289910292480,"available":288341602304,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2690,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.444254"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20874,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087299+0000","last_peered":"2024-09-05T13:23:22.087299+0000","last_clean":"2024-09-05T13:23:22.087299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087299+0000","last_undegraded":"2024-09-05T13:23:22.087299+0000","last_fullsized":"2024-09-05T13:23:22.087299+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21111,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086964+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086964+0000","last_peered":"2024-09-05T13:23:22.086964+0000","last_clean":"2024-09-05T13:23:22.086964+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086964+0000","last_undegraded":"2024-09-05T13:23:22.086964+0000","last_fullsized":"2024-09-05T13:23:22.086964+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":29926,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:39.126152+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:39.126152+0000","last_peered":"2024-09-05T13:23:39.126152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:39.126152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4343,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-112}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22232,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483372,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888468,"statfs":{"total":96636764160,"available":96141791232,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:40.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:40.707 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:40.707 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:40.722 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-05T13:23:40.201898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22233,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531924,"kb_used_data":215196,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583596,"statfs":{"total":289910292480,"available":288341602304,"internally_reserved":0,"allocated":220360704,"data_stored":219444671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2690,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.444254"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21062,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082564+0000","last_change":"2024-09-05T13:13:18.492502+0000","last_active":"2024-09-05T13:23:22.082564+0000","last_peered":"2024-09-05T13:23:22.082564+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:22.082564+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20874,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087299+0000","last_peered":"2024-09-05T13:23:22.087299+0000","last_clean":"2024-09-05T13:23:22.087299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087299+0000","last_undegraded":"2024-09-05T13:23:22.087299+0000","last_fullsized":"2024-09-05T13:23:22.087299+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079325+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:22.079325+0000","last_peered":"2024-09-05T13:23:22.079325+0000","last_clean":"2024-09-05T13:23:22.079325+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:22.079325+0000","last_undegraded":"2024-09-05T13:23:22.079325+0000","last_fullsized":"2024-09-05T13:23:22.079325+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21111,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086964+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086964+0000","last_peered":"2024-09-05T13:23:22.086964+0000","last_clean":"2024-09-05T13:23:22.086964+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086964+0000","last_undegraded":"2024-09-05T13:23:22.086964+0000","last_fullsized":"2024-09-05T13:23:22.086964+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21032,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080282+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:22.080282+0000","last_peered":"2024-09-05T13:23:22.080282+0000","last_clean":"2024-09-05T13:23:22.080282+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:22.080282+0000","last_undegraded":"2024-09-05T13:23:22.080282+0000","last_fullsized":"2024-09-05T13:23:22.080282+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21308,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080501+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:22.080501+0000","last_peered":"2024-09-05T13:23:22.080501+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:22.080501+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29669,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078402+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:22.078402+0000","last_peered":"2024-09-05T13:23:22.078402+0000","last_clean":"2024-09-05T13:23:22.078402+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:22.078402+0000","last_undegraded":"2024-09-05T13:23:22.078402+0000","last_fullsized":"2024-09-05T13:23:22.078402+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21251,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080176+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:22.080176+0000","last_peered":"2024-09-05T13:23:22.080176+0000","last_clean":"2024-09-05T13:23:22.080176+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:22.080176+0000","last_undegraded":"2024-09-05T13:23:22.080176+0000","last_fullsized":"2024-09-05T13:23:22.080176+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078172+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:22.078172+0000","last_peered":"2024-09-05T13:23:22.078172+0000","last_clean":"2024-09-05T13:23:22.078172+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:22.078172+0000","last_undegraded":"2024-09-05T13:23:22.078172+0000","last_fullsized":"2024-09-05T13:23:22.078172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":29926,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:39.126152+0000","last_change":"2024-09-05T13:23:20.386406+0000","last_active":"2024-09-05T13:23:39.126152+0000","last_peered":"2024-09-05T13:23:39.126152+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:39.126152+0000","last_undegraded":"2024-09-05T13:13:17.951355+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":42,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4343,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-112}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080709+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:22.080709+0000","last_peered":"2024-09-05T13:23:22.080709+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:22.080709+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":119,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.080396+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:22.080396+0000","last_peered":"2024-09-05T13:23:22.080396+0000","last_clean":"2024-09-05T13:23:22.080396+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:22.080396+0000","last_undegraded":"2024-09-05T13:23:22.080396+0000","last_fullsized":"2024-09-05T13:23:22.080396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.079201+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:22.079201+0000","last_peered":"2024-09-05T13:23:22.079201+0000","last_clean":"2024-09-05T13:23:22.079201+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:22.079201+0000","last_undegraded":"2024-09-05T13:23:22.079201+0000","last_fullsized":"2024-09-05T13:23:22.079201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082219+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:22.082219+0000","last_peered":"2024-09-05T13:23:22.082219+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:22.082219+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078782+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:22.078782+0000","last_peered":"2024-09-05T13:23:22.078782+0000","last_clean":"2024-09-05T13:23:22.078782+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:22.078782+0000","last_undegraded":"2024-09-05T13:23:22.078782+0000","last_fullsized":"2024-09-05T13:23:22.078782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21306,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.082210+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:22.082210+0000","last_peered":"2024-09-05T13:23:22.082210+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:22.082210+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.189554+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:22.189554+0000","last_peered":"2024-09-05T13:23:22.189554+0000","last_clean":"2024-09-05T13:23:22.189554+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:22.189554+0000","last_undegraded":"2024-09-05T13:23:22.189554+0000","last_fullsized":"2024-09-05T13:23:22.189554+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078440+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:22.078440+0000","last_peered":"2024-09-05T13:23:22.078440+0000","last_clean":"2024-09-05T13:23:22.078440+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:22.078440+0000","last_undegraded":"2024-09-05T13:23:22.078440+0000","last_fullsized":"2024-09-05T13:23:22.078440+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.078622+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:22.078622+0000","last_peered":"2024-09-05T13:23:22.078622+0000","last_clean":"2024-09-05T13:23:22.078622+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:22.078622+0000","last_undegraded":"2024-09-05T13:23:22.078622+0000","last_fullsized":"2024-09-05T13:23:22.078622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21347,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.080380+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:22.080380+0000","last_peered":"2024-09-05T13:23:22.080380+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:22.080380+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31266,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22232,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562440,"kb_used_data":71284,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809400,"statfs":{"total":96636764160,"available":96060825600,"internally_reserved":0,"allocated":72994816,"data_stored":72703769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483372,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888468,"statfs":{"total":96636764160,"available":96141791232,"internally_reserved":0,"allocated":487424,"data_stored":149408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:40.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:40.723 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21062, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082564+0000', 'last_change': '2024-09-05T13:13:18.492502+0000', 'last_active': '2024-09-05T13:23:22.082564+0000', 'last_peered': '2024-09-05T13:23:22.082564+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:22.082564+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4421, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4421}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21308, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080501+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:22.080501+0000', 'last_peered': '2024-09-05T13:23:22.080501+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:22.080501+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 29926, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:39.126152+0000', 'last_change': '2024-09-05T13:23:20.386406+0000', 'last_active': '2024-09-05T13:23:39.126152+0000', 'last_peered': '2024-09-05T13:23:39.126152+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:23:39.126152+0000', 'last_undegraded': '2024-09-05T13:13:17.951355+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 42, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 112, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4343, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -112}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21063, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080709+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:22.080709+0000', 'last_peered': '2024-09-05T13:23:22.080709+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:22.080709+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20720, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.086353+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:22.086353+0000', 'last_peered': '2024-09-05T13:23:22.086353+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:22.086353+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:40.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21033, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082219+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:22.082219+0000', 'last_peered': '2024-09-05T13:23:22.082219+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:22.082219+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:40.725 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:40.725 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21306, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.082210+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:22.082210+0000', 'last_peered': '2024-09-05T13:23:22.082210+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:22.082210+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:40.725 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:40.725 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21347, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.080380+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:22.080380+0000', 'last_peered': '2024-09-05T13:23:22.080380+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:22.080380+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:40.725 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:41.162 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:41.162 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31266/6 objects degraded (521100.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":748,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1568690176,"bytes_avail":288341602304,"bytes_total":289910292480,"degraded_objects":31266,"degraded_total":6,"degraded_ratio":5211,"recovering_objects_per_sec":310,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":3728,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:41.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:41.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:41.514 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:41.514 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:41.527 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-05T13:23:41.199283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22574,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544424,"kb_used_data":215212,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281571096,"statfs":{"total":289910292480,"available":288328802304,"internally_reserved":0,"allocated":220377088,"data_stored":219450481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1361038342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3031,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.441215"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21523,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.191116+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:41.191116+0000","last_peered":"2024-09-05T13:23:41.191116+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:41.191116+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20874,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087299+0000","last_peered":"2024-09-05T13:23:22.087299+0000","last_clean":"2024-09-05T13:23:22.087299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087299+0000","last_undegraded":"2024-09-05T13:23:22.087299+0000","last_fullsized":"2024-09-05T13:23:22.087299+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28591,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202932+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:40.202932+0000","last_peered":"2024-09-05T13:23:40.202932+0000","last_clean":"2024-09-05T13:23:40.202932+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:40.202932+0000","last_undegraded":"2024-09-05T13:23:40.202932+0000","last_fullsized":"2024-09-05T13:23:40.202932+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21111,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086964+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086964+0000","last_peered":"2024-09-05T13:23:22.086964+0000","last_clean":"2024-09-05T13:23:22.086964+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086964+0000","last_undegraded":"2024-09-05T13:23:22.086964+0000","last_fullsized":"2024-09-05T13:23:22.086964+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21034,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205771+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:40.205771+0000","last_peered":"2024-09-05T13:23:40.205771+0000","last_clean":"2024-09-05T13:23:40.205771+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:40.205771+0000","last_undegraded":"2024-09-05T13:23:40.205771+0000","last_fullsized":"2024-09-05T13:23:40.205771+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.205680+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:40.205680+0000","last_peered":"2024-09-05T13:23:40.205680+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:40.205680+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29671,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202915+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:40.202915+0000","last_peered":"2024-09-05T13:23:40.202915+0000","last_clean":"2024-09-05T13:23:40.202915+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:40.202915+0000","last_undegraded":"2024-09-05T13:23:40.202915+0000","last_fullsized":"2024-09-05T13:23:40.202915+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21253,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205288+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:40.205288+0000","last_peered":"2024-09-05T13:23:40.205288+0000","last_clean":"2024-09-05T13:23:40.205288+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:40.205288+0000","last_undegraded":"2024-09-05T13:23:40.205288+0000","last_fullsized":"2024-09-05T13:23:40.205288+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202850+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:40.202850+0000","last_peered":"2024-09-05T13:23:40.202850+0000","last_clean":"2024-09-05T13:23:40.202850+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:40.202850+0000","last_undegraded":"2024-09-05T13:23:40.202850+0000","last_fullsized":"2024-09-05T13:23:40.202850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21237,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-05T13:23:39.452308+0000","last_change":"2024-09-05T13:23:40.751508+0000","last_active":"2024-09-05T13:23:40.751508+0000","last_peered":"2024-09-05T13:23:39.452308+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:39.452308+0000","last_undegraded":"2024-09-05T13:23:39.452308+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21065,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206911+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:40.206911+0000","last_peered":"2024-09-05T13:23:40.206911+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:40.206911+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":121,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205578+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:40.205578+0000","last_peered":"2024-09-05T13:23:40.205578+0000","last_clean":"2024-09-05T13:23:40.205578+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:40.205578+0000","last_undegraded":"2024-09-05T13:23:40.205578+0000","last_fullsized":"2024-09-05T13:23:40.205578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203013+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:40.203013+0000","last_peered":"2024-09-05T13:23:40.203013+0000","last_clean":"2024-09-05T13:23:40.203013+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:40.203013+0000","last_undegraded":"2024-09-05T13:23:40.203013+0000","last_fullsized":"2024-09-05T13:23:40.203013+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206182+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:40.206182+0000","last_peered":"2024-09-05T13:23:40.206182+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:40.206182+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203107+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:40.203107+0000","last_peered":"2024-09-05T13:23:40.203107+0000","last_clean":"2024-09-05T13:23:40.203107+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:40.203107+0000","last_undegraded":"2024-09-05T13:23:40.203107+0000","last_fullsized":"2024-09-05T13:23:40.203107+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21308,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206703+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:40.206703+0000","last_peered":"2024-09-05T13:23:40.206703+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:40.206703+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21241,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203144+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:40.203144+0000","last_peered":"2024-09-05T13:23:40.203144+0000","last_clean":"2024-09-05T13:23:40.203144+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:40.203144+0000","last_undegraded":"2024-09-05T13:23:40.203144+0000","last_fullsized":"2024-09-05T13:23:40.203144+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203258+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:40.203258+0000","last_peered":"2024-09-05T13:23:40.203258+0000","last_clean":"2024-09-05T13:23:40.203258+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:40.203258+0000","last_undegraded":"2024-09-05T13:23:40.203258+0000","last_fullsized":"2024-09-05T13:23:40.203258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203629+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:40.203629+0000","last_peered":"2024-09-05T13:23:40.203629+0000","last_clean":"2024-09-05T13:23:40.203629+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:40.203629+0000","last_undegraded":"2024-09-05T13:23:40.203629+0000","last_fullsized":"2024-09-05T13:23:40.203629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21349,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206284+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:40.206284+0000","last_peered":"2024-09-05T13:23:40.206284+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:40.206284+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22573,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562448,"kb_used_data":71292,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809392,"statfs":{"total":96636764160,"available":96060817408,"internally_reserved":0,"allocated":73003008,"data_stored":72706674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495864,"kb_used_data":484,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875976,"statfs":{"total":96636764160,"available":96128999424,"internally_reserved":0,"allocated":495616,"data_stored":152313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":507244207},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:44.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:44.863 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:44.864 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:44.876 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-09-05T13:23:43.760184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22574,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544424,"kb_used_data":215212,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281571096,"statfs":{"total":289910292480,"available":288328802304,"internally_reserved":0,"allocated":220377088,"data_stored":219450481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1361038342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1782,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001389"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21523,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.191116+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:41.191116+0000","last_peered":"2024-09-05T13:23:41.191116+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:41.191116+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20874,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087299+0000","last_peered":"2024-09-05T13:23:22.087299+0000","last_clean":"2024-09-05T13:23:22.087299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087299+0000","last_undegraded":"2024-09-05T13:23:22.087299+0000","last_fullsized":"2024-09-05T13:23:22.087299+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28591,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202932+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:40.202932+0000","last_peered":"2024-09-05T13:23:40.202932+0000","last_clean":"2024-09-05T13:23:40.202932+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:40.202932+0000","last_undegraded":"2024-09-05T13:23:40.202932+0000","last_fullsized":"2024-09-05T13:23:40.202932+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21111,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086964+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086964+0000","last_peered":"2024-09-05T13:23:22.086964+0000","last_clean":"2024-09-05T13:23:22.086964+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086964+0000","last_undegraded":"2024-09-05T13:23:22.086964+0000","last_fullsized":"2024-09-05T13:23:22.086964+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21034,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205771+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:40.205771+0000","last_peered":"2024-09-05T13:23:40.205771+0000","last_clean":"2024-09-05T13:23:40.205771+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:40.205771+0000","last_undegraded":"2024-09-05T13:23:40.205771+0000","last_fullsized":"2024-09-05T13:23:40.205771+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.205680+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:40.205680+0000","last_peered":"2024-09-05T13:23:40.205680+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:40.205680+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29671,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202915+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:40.202915+0000","last_peered":"2024-09-05T13:23:40.202915+0000","last_clean":"2024-09-05T13:23:40.202915+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:40.202915+0000","last_undegraded":"2024-09-05T13:23:40.202915+0000","last_fullsized":"2024-09-05T13:23:40.202915+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21253,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205288+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:40.205288+0000","last_peered":"2024-09-05T13:23:40.205288+0000","last_clean":"2024-09-05T13:23:40.205288+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:40.205288+0000","last_undegraded":"2024-09-05T13:23:40.205288+0000","last_fullsized":"2024-09-05T13:23:40.205288+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202850+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:40.202850+0000","last_peered":"2024-09-05T13:23:40.202850+0000","last_clean":"2024-09-05T13:23:40.202850+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:40.202850+0000","last_undegraded":"2024-09-05T13:23:40.202850+0000","last_fullsized":"2024-09-05T13:23:40.202850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21237,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-05T13:23:39.452308+0000","last_change":"2024-09-05T13:23:40.751508+0000","last_active":"2024-09-05T13:23:40.751508+0000","last_peered":"2024-09-05T13:23:39.452308+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:39.452308+0000","last_undegraded":"2024-09-05T13:23:39.452308+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21065,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206911+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:40.206911+0000","last_peered":"2024-09-05T13:23:40.206911+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:40.206911+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":121,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205578+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:40.205578+0000","last_peered":"2024-09-05T13:23:40.205578+0000","last_clean":"2024-09-05T13:23:40.205578+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:40.205578+0000","last_undegraded":"2024-09-05T13:23:40.205578+0000","last_fullsized":"2024-09-05T13:23:40.205578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203013+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:40.203013+0000","last_peered":"2024-09-05T13:23:40.203013+0000","last_clean":"2024-09-05T13:23:40.203013+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:40.203013+0000","last_undegraded":"2024-09-05T13:23:40.203013+0000","last_fullsized":"2024-09-05T13:23:40.203013+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206182+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:40.206182+0000","last_peered":"2024-09-05T13:23:40.206182+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:40.206182+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203107+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:40.203107+0000","last_peered":"2024-09-05T13:23:40.203107+0000","last_clean":"2024-09-05T13:23:40.203107+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:40.203107+0000","last_undegraded":"2024-09-05T13:23:40.203107+0000","last_fullsized":"2024-09-05T13:23:40.203107+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21308,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206703+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:40.206703+0000","last_peered":"2024-09-05T13:23:40.206703+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:40.206703+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21241,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203144+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:40.203144+0000","last_peered":"2024-09-05T13:23:40.203144+0000","last_clean":"2024-09-05T13:23:40.203144+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:40.203144+0000","last_undegraded":"2024-09-05T13:23:40.203144+0000","last_fullsized":"2024-09-05T13:23:40.203144+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203258+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:40.203258+0000","last_peered":"2024-09-05T13:23:40.203258+0000","last_clean":"2024-09-05T13:23:40.203258+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:40.203258+0000","last_undegraded":"2024-09-05T13:23:40.203258+0000","last_fullsized":"2024-09-05T13:23:40.203258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203629+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:40.203629+0000","last_peered":"2024-09-05T13:23:40.203629+0000","last_clean":"2024-09-05T13:23:40.203629+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:40.203629+0000","last_undegraded":"2024-09-05T13:23:40.203629+0000","last_fullsized":"2024-09-05T13:23:40.203629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21349,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206284+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:40.206284+0000","last_peered":"2024-09-05T13:23:40.206284+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:40.206284+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22573,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562448,"kb_used_data":71292,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809392,"statfs":{"total":96636764160,"available":96060817408,"internally_reserved":0,"allocated":73003008,"data_stored":72706674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495864,"kb_used_data":484,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875976,"statfs":{"total":96636764160,"available":96128999424,"internally_reserved":0,"allocated":495616,"data_stored":152313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":507244207},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:44.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:45.212 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:45.213 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:45.226 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-09-05T13:23:43.760184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22574,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544424,"kb_used_data":215212,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281571096,"statfs":{"total":289910292480,"available":288328802304,"internally_reserved":0,"allocated":220377088,"data_stored":219450481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1361038342},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1782,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001389"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21523,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.191116+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:41.191116+0000","last_peered":"2024-09-05T13:23:41.191116+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:41.191116+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20874,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087299+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:22.087299+0000","last_peered":"2024-09-05T13:23:22.087299+0000","last_clean":"2024-09-05T13:23:22.087299+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:22.087299+0000","last_undegraded":"2024-09-05T13:23:22.087299+0000","last_fullsized":"2024-09-05T13:23:22.087299+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28591,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202932+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:40.202932+0000","last_peered":"2024-09-05T13:23:40.202932+0000","last_clean":"2024-09-05T13:23:40.202932+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:40.202932+0000","last_undegraded":"2024-09-05T13:23:40.202932+0000","last_fullsized":"2024-09-05T13:23:40.202932+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21111,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086964+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:22.086964+0000","last_peered":"2024-09-05T13:23:22.086964+0000","last_clean":"2024-09-05T13:23:22.086964+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:22.086964+0000","last_undegraded":"2024-09-05T13:23:22.086964+0000","last_fullsized":"2024-09-05T13:23:22.086964+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21034,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205771+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:40.205771+0000","last_peered":"2024-09-05T13:23:40.205771+0000","last_clean":"2024-09-05T13:23:40.205771+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:40.205771+0000","last_undegraded":"2024-09-05T13:23:40.205771+0000","last_fullsized":"2024-09-05T13:23:40.205771+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.205680+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:40.205680+0000","last_peered":"2024-09-05T13:23:40.205680+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:40.205680+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29671,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202915+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:40.202915+0000","last_peered":"2024-09-05T13:23:40.202915+0000","last_clean":"2024-09-05T13:23:40.202915+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:40.202915+0000","last_undegraded":"2024-09-05T13:23:40.202915+0000","last_fullsized":"2024-09-05T13:23:40.202915+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21253,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205288+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:40.205288+0000","last_peered":"2024-09-05T13:23:40.205288+0000","last_clean":"2024-09-05T13:23:40.205288+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:40.205288+0000","last_undegraded":"2024-09-05T13:23:40.205288+0000","last_fullsized":"2024-09-05T13:23:40.205288+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.202850+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:40.202850+0000","last_peered":"2024-09-05T13:23:40.202850+0000","last_clean":"2024-09-05T13:23:40.202850+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:40.202850+0000","last_undegraded":"2024-09-05T13:23:40.202850+0000","last_fullsized":"2024-09-05T13:23:40.202850+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21237,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-05T13:23:39.452308+0000","last_change":"2024-09-05T13:23:40.751508+0000","last_active":"2024-09-05T13:23:40.751508+0000","last_peered":"2024-09-05T13:23:39.452308+0000","last_clean":"2024-09-05T13:13:15.856125+0000","last_became_active":"2024-09-05T13:13:18.487022+0000","last_became_peered":"2024-09-05T13:13:18.487022+0000","last_unstale":"2024-09-05T13:23:39.452308+0000","last_undegraded":"2024-09-05T13:23:39.452308+0000","last_fullsized":"2024-09-05T13:13:17.904688+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086406+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:22.086406+0000","last_peered":"2024-09-05T13:23:22.086406+0000","last_clean":"2024-09-05T13:23:22.086406+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:22.086406+0000","last_undegraded":"2024-09-05T13:23:22.086406+0000","last_fullsized":"2024-09-05T13:23:22.086406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21065,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206911+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:40.206911+0000","last_peered":"2024-09-05T13:23:40.206911+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:40.206911+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":121,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205578+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:40.205578+0000","last_peered":"2024-09-05T13:23:40.205578+0000","last_clean":"2024-09-05T13:23:40.205578+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:40.205578+0000","last_undegraded":"2024-09-05T13:23:40.205578+0000","last_fullsized":"2024-09-05T13:23:40.205578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29827,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.086480+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:22.086480+0000","last_peered":"2024-09-05T13:23:22.086480+0000","last_clean":"2024-09-05T13:23:22.086480+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:22.086480+0000","last_undegraded":"2024-09-05T13:23:22.086480+0000","last_fullsized":"2024-09-05T13:23:22.086480+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203013+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:40.203013+0000","last_peered":"2024-09-05T13:23:40.203013+0000","last_clean":"2024-09-05T13:23:40.203013+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:40.203013+0000","last_undegraded":"2024-09-05T13:23:40.203013+0000","last_fullsized":"2024-09-05T13:23:40.203013+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20720,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:22.086353+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:22.086353+0000","last_peered":"2024-09-05T13:23:22.086353+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:22.086353+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206182+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:40.206182+0000","last_peered":"2024-09-05T13:23:40.206182+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:40.206182+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203107+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:40.203107+0000","last_peered":"2024-09-05T13:23:40.203107+0000","last_clean":"2024-09-05T13:23:40.203107+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:40.203107+0000","last_undegraded":"2024-09-05T13:23:40.203107+0000","last_fullsized":"2024-09-05T13:23:40.203107+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21308,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206703+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:40.206703+0000","last_peered":"2024-09-05T13:23:40.206703+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:40.206703+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-05T13:23:22.087060+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:22.087060+0000","last_peered":"2024-09-05T13:23:22.087060+0000","last_clean":"2024-09-05T13:23:22.087060+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:22.087060+0000","last_undegraded":"2024-09-05T13:23:22.087060+0000","last_fullsized":"2024-09-05T13:23:22.087060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21241,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203144+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:40.203144+0000","last_peered":"2024-09-05T13:23:40.203144+0000","last_clean":"2024-09-05T13:23:40.203144+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:40.203144+0000","last_undegraded":"2024-09-05T13:23:40.203144+0000","last_fullsized":"2024-09-05T13:23:40.203144+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203258+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:40.203258+0000","last_peered":"2024-09-05T13:23:40.203258+0000","last_clean":"2024-09-05T13:23:40.203258+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:40.203258+0000","last_undegraded":"2024-09-05T13:23:40.203258+0000","last_fullsized":"2024-09-05T13:23:40.203258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.203629+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:40.203629+0000","last_peered":"2024-09-05T13:23:40.203629+0000","last_clean":"2024-09-05T13:23:40.203629+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:40.203629+0000","last_undegraded":"2024-09-05T13:23:40.203629+0000","last_fullsized":"2024-09-05T13:23:40.203629+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21349,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206284+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:40.206284+0000","last_peered":"2024-09-05T13:23:40.206284+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:40.206284+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22573,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486112,"kb_used_data":143436,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885728,"statfs":{"total":96636764160,"available":96138985472,"internally_reserved":0,"allocated":146878464,"data_stored":146591494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562448,"kb_used_data":71292,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809392,"statfs":{"total":96636764160,"available":96060817408,"internally_reserved":0,"allocated":73003008,"data_stored":72706674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495864,"kb_used_data":484,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875976,"statfs":{"total":96636764160,"available":96128999424,"internally_reserved":0,"allocated":495616,"data_stored":152313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":507244207},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 21523, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.191116+0000', 'last_change': '2024-09-05T13:23:39.452618+0000', 'last_active': '2024-09-05T13:23:41.191116+0000', 'last_peered': '2024-09-05T13:23:41.191116+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:41.191116+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4192, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 229, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4192}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21310, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:40.205680+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:40.205680+0000', 'last_peered': '2024-09-05T13:23:40.205680+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:40.205680+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'21093", 'reported_seq': 21237, 'reported_epoch': 52, 'state': 'peering', 'last_fresh': '2024-09-05T13:23:39.452308+0000', 'last_change': '2024-09-05T13:23:40.751508+0000', 'last_active': '2024-09-05T13:23:40.751508+0000', 'last_peered': '2024-09-05T13:23:39.452308+0000', 'last_clean': '2024-09-05T13:13:15.856125+0000', 'last_became_active': '2024-09-05T13:13:18.487022+0000', 'last_became_peered': '2024-09-05T13:13:18.487022+0000', 'last_unstale': '2024-09-05T13:23:39.452308+0000', 'last_undegraded': '2024-09-05T13:23:39.452308+0000', 'last_fullsized': '2024-09-05T13:13:17.904688+0000', 'mapping_epoch': 52, 'log_start': "45'11000", 'ondisk_log_start': "45'11000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2383", 'last_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:49.049957+0000', 'objects_scrubbed': 2339, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 916, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18747, 'num_write_kb': 32816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4455, 'num_bytes_recovered': 8192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21065, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:40.206911+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:40.206911+0000', 'last_peered': '2024-09-05T13:23:40.206911+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:40.206911+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:45.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:45.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20720, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:22.086353+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:22.086353+0000', 'last_peered': '2024-09-05T13:23:22.086353+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:22.086353+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:45.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:45.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21035, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:40.206182+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:40.206182+0000', 'last_peered': '2024-09-05T13:23:40.206182+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:40.206182+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:45.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:45.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21308, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:40.206703+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:40.206703+0000', 'last_peered': '2024-09-05T13:23:40.206703+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:40.206703+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:45.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:45.229 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21349, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:40.206284+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:40.206284+0000', 'last_peered': '2024-09-05T13:23:40.206284+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:40.206284+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:45.229 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:45.644 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:45.644 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30925/6 objects degraded (515416.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":752,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1581490176,"bytes_avail":288328802304,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":30925,"degraded_total":6,"degraded_ratio":5154.166666666667,"recovering_objects_per_sec":222,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1782,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:45.657 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:45.658 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:45.981 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:45.996 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-05T13:23:45.760546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22574,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531968,"kb_used_data":215236,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583552,"statfs":{"total":289910292480,"available":288341557248,"internally_reserved":0,"allocated":220401664,"data_stored":219459154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1782,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001368"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":21523,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.191116+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:41.191116+0000","last_peered":"2024-09-05T13:23:41.191116+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:41.191116+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21034,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205771+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:40.205771+0000","last_peered":"2024-09-05T13:23:40.205771+0000","last_clean":"2024-09-05T13:23:40.205771+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:40.205771+0000","last_undegraded":"2024-09-05T13:23:40.205771+0000","last_fullsized":"2024-09-05T13:23:40.205771+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.205680+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:40.205680+0000","last_peered":"2024-09-05T13:23:40.205680+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:40.205680+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21253,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205288+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:40.205288+0000","last_peered":"2024-09-05T13:23:40.205288+0000","last_clean":"2024-09-05T13:23:40.205288+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:40.205288+0000","last_undegraded":"2024-09-05T13:23:40.205288+0000","last_fullsized":"2024-09-05T13:23:40.205288+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21065,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206911+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:40.206911+0000","last_peered":"2024-09-05T13:23:40.206911+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:40.206911+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":121,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-05T13:23:40.205578+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:40.205578+0000","last_peered":"2024-09-05T13:23:40.205578+0000","last_clean":"2024-09-05T13:23:40.205578+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:40.205578+0000","last_undegraded":"2024-09-05T13:23:40.205578+0000","last_fullsized":"2024-09-05T13:23:40.205578+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206182+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:40.206182+0000","last_peered":"2024-09-05T13:23:40.206182+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:40.206182+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21308,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206703+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:40.206703+0000","last_peered":"2024-09-05T13:23:40.206703+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:40.206703+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202401+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202401+0000","last_peered":"2024-09-05T13:23:41.202401+0000","last_clean":"2024-09-05T13:23:41.202401+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202401+0000","last_undegraded":"2024-09-05T13:23:41.202401+0000","last_fullsized":"2024-09-05T13:23:41.202401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21349,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:40.206284+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:40.206284+0000","last_peered":"2024-09-05T13:23:40.206284+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:40.206284+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30925,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22573,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562448,"kb_used_data":71292,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809392,"statfs":{"total":96636764160,"available":96060817408,"internally_reserved":0,"allocated":73003008,"data_stored":72706674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:48.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:49.386 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:49.386 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:49.398 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-05T13:23:47.760871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23532,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531980,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583540,"statfs":{"total":289910292480,"available":288341544960,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001350"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":23441,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:46.195399+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:46.195399+0000","last_peered":"2024-09-05T13:23:46.195399+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:46.195399+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21312,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204044+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.204044+0000","last_peered":"2024-09-05T13:23:41.204044+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.204044+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21310,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203640+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.203640+0000","last_peered":"2024-09-05T13:23:41.203640+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.203640+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202401+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202401+0000","last_peered":"2024-09-05T13:23:41.202401+0000","last_clean":"2024-09-05T13:23:41.202401+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202401+0000","last_undegraded":"2024-09-05T13:23:41.202401+0000","last_fullsized":"2024-09-05T13:23:41.202401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23531,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:49.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:49.747 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:49.747 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:49.759 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-05T13:23:47.760871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23532,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531980,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583540,"statfs":{"total":289910292480,"available":288341544960,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001350"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":23441,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:46.195399+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:46.195399+0000","last_peered":"2024-09-05T13:23:46.195399+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:46.195399+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21312,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204044+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.204044+0000","last_peered":"2024-09-05T13:23:41.204044+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.204044+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21310,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203640+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.203640+0000","last_peered":"2024-09-05T13:23:41.203640+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.203640+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202401+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202401+0000","last_peered":"2024-09-05T13:23:41.202401+0000","last_clean":"2024-09-05T13:23:41.202401+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202401+0000","last_undegraded":"2024-09-05T13:23:41.202401+0000","last_fullsized":"2024-09-05T13:23:41.202401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23531,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:49.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 23441, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:46.195399+0000', 'last_change': '2024-09-05T13:23:39.452618+0000', 'last_active': '2024-09-05T13:23:46.195399+0000', 'last_peered': '2024-09-05T13:23:46.195399+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:46.195399+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3234, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1187, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3234}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21312, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204044+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:41.204044+0000', 'last_peered': '2024-09-05T13:23:41.204044+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:41.204044+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21067, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204706+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:41.204706+0000', 'last_peered': '2024-09-05T13:23:41.204706+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:41.204706+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20724, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204319+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:41.204319+0000', 'last_peered': '2024-09-05T13:23:41.204319+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:41.204319+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:49.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21037, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204660+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:41.204660+0000', 'last_peered': '2024-09-05T13:23:41.204660+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:41.204660+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:49.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:49.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21310, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.203640+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:41.203640+0000', 'last_peered': '2024-09-05T13:23:41.203640+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:41.203640+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:49.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:49.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21351, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.203415+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:41.203415+0000', 'last_peered': '2024-09-05T13:23:41.203415+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:41.203415+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:49.762 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:50.134 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:50.134 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29967/6 objects degraded (499450.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":757,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1568747520,"bytes_avail":288341544960,"bytes_total":289910292480,"degraded_objects":29967,"degraded_total":6,"degraded_ratio":4994.5,"recovering_objects_per_sec":162,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1299,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:50.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:50.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:50.499 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:50.499 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:50.513 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-05T13:23:49.761214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23532,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531980,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583540,"statfs":{"total":289910292480,"available":288341544960,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.559315"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":23441,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:46.195399+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:46.195399+0000","last_peered":"2024-09-05T13:23:46.195399+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:46.195399+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3234}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21312,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204044+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.204044+0000","last_peered":"2024-09-05T13:23:41.204044+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.204044+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21310,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203640+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.203640+0000","last_peered":"2024-09-05T13:23:41.203640+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.203640+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202401+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202401+0000","last_peered":"2024-09-05T13:23:41.202401+0000","last_clean":"2024-09-05T13:23:41.202401+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202401+0000","last_undegraded":"2024-09-05T13:23:41.202401+0000","last_fullsized":"2024-09-05T13:23:41.202401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23531,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:53.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:53.888 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:53.888 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:53.901 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-05T13:23:53.761950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24531,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540300,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575220,"statfs":{"total":289910292480,"available":288333025280,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1957,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002072"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":25439,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:51.196980+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:51.196980+0000","last_peered":"2024-09-05T13:23:51.196980+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:51.196980+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21312,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204044+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.204044+0000","last_peered":"2024-09-05T13:23:41.204044+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.204044+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24530,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491708,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880132,"statfs":{"total":96636764160,"available":96133255168,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:53.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:54.214 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:54.214 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:54.229 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-05T13:23:53.761950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24531,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540300,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575220,"statfs":{"total":289910292480,"available":288333025280,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1957,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002072"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":25439,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:51.196980+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:51.196980+0000","last_peered":"2024-09-05T13:23:51.196980+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:51.196980+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21312,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204044+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.204044+0000","last_peered":"2024-09-05T13:23:41.204044+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.204044+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24530,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491708,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880132,"statfs":{"total":96636764160,"available":96133255168,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:54.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:54.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 25439, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:51.196980+0000', 'last_change': '2024-09-05T13:23:39.452618+0000', 'last_active': '2024-09-05T13:23:51.196980+0000', 'last_peered': '2024-09-05T13:23:51.196980+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:51.196980+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2235, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2186, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2235}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:54.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:54.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21312, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204044+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:41.204044+0000', 'last_peered': '2024-09-05T13:23:41.204044+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:41.204044+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:54.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:54.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21067, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204706+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:41.204706+0000', 'last_peered': '2024-09-05T13:23:41.204706+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:41.204706+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:54.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:54.231 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20724, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204319+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:41.204319+0000', 'last_peered': '2024-09-05T13:23:41.204319+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:41.204319+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:54.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:54.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21037, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204660+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:41.204660+0000', 'last_peered': '2024-09-05T13:23:41.204660+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:41.204660+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:54.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:54.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21311, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.205116+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:41.205116+0000', 'last_peered': '2024-09-05T13:23:41.205116+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:41.205116+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:54.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:54.232 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21351, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.203415+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:41.203415+0000', 'last_peered': '2024-09-05T13:23:41.203415+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:41.203415+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:54.232 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:54.622 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28968/6 objects degraded (482800.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":761,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1577267200,"bytes_avail":288333025280,"bytes_total":289910292480,"degraded_objects":28968,"degraded_total":6,"degraded_ratio":4828,"recovering_objects_per_sec":163,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1957,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:54.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:54.635 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:54.935 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:54.935 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:54.947 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-05T13:23:53.761950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24531,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540300,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575220,"statfs":{"total":289910292480,"available":288333025280,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1957,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1957,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002072"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":25439,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:51.196980+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:51.196980+0000","last_peered":"2024-09-05T13:23:51.196980+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:51.196980+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2235,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21312,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204044+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.204044+0000","last_peered":"2024-09-05T13:23:41.204044+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.204044+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24530,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491708,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880132,"statfs":{"total":96636764160,"available":96133255168,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:57.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:58.280 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:58.294 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-05T13:23:57.762572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25347,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531980,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583540,"statfs":{"total":289910292480,"available":288341544960,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2773,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002025"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":27071,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:56.197382+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:56.197382+0000","last_peered":"2024-09-05T13:23:56.197382+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:56.197382+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3002,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21313,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205482+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.205482+0000","last_peered":"2024-09-05T13:23:41.205482+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.205482+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25346,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:58.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:58.607 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:58.607 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:58.620 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-05T13:23:57.762572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25347,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531980,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583540,"statfs":{"total":289910292480,"available":288341544960,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2773,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002025"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":27071,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:56.197382+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:56.197382+0000","last_peered":"2024-09-05T13:23:56.197382+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:56.197382+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3002,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21313,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205482+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.205482+0000","last_peered":"2024-09-05T13:23:41.205482+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.205482+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25346,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:23:58.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:23:58.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 27071, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:56.197382+0000', 'last_change': '2024-09-05T13:23:39.452618+0000', 'last_active': '2024-09-05T13:23:56.197382+0000', 'last_peered': '2024-09-05T13:23:56.197382+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:23:56.197382+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1419, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3002, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1419}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:58.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21313, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.205482+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:41.205482+0000', 'last_peered': '2024-09-05T13:23:41.205482+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:41.205482+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21067, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204706+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:41.204706+0000', 'last_peered': '2024-09-05T13:23:41.204706+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:41.204706+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20724, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204319+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:41.204319+0000', 'last_peered': '2024-09-05T13:23:41.204319+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:41.204319+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21037, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204660+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:41.204660+0000', 'last_peered': '2024-09-05T13:23:41.204660+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:41.204660+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21311, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.205116+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:41.205116+0000', 'last_peered': '2024-09-05T13:23:41.205116+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:41.205116+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:23:58.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21351, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.203415+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:41.203415+0000', 'last_peered': '2024-09-05T13:23:41.203415+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:41.203415+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:23:58.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:23:59.033 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:59.033 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28152/6 objects degraded (469200.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":765,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1568747520,"bytes_avail":288341544960,"bytes_total":289910292480,"degraded_objects":28152,"degraded_total":6,"degraded_ratio":4692,"recovering_objects_per_sec":231,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2773,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:23:59.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:23:59.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:23:59.374 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:23:59.374 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:23:59.386 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-05T13:23:57.762572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25347,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531980,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583540,"statfs":{"total":289910292480,"available":288341544960,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2773,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002025"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":27071,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:56.197382+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:23:56.197382+0000","last_peered":"2024-09-05T13:23:56.197382+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:23:56.197382+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3002,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1419}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21313,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205482+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.205482+0000","last_peered":"2024-09-05T13:23:41.205482+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.205482+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29673,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201993+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.201993+0000","last_peered":"2024-09-05T13:23:41.201993+0000","last_clean":"2024-09-05T13:23:41.201993+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.201993+0000","last_undegraded":"2024-09-05T13:23:41.201993+0000","last_fullsized":"2024-09-05T13:23:41.201993+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21037,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204660+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.204660+0000","last_peered":"2024-09-05T13:23:41.204660+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.204660+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28152,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25346,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483388,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888452,"statfs":{"total":96636764160,"available":96141774848,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:02.388 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:02.713 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:02.714 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:02.726 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-05T13:24:02.261910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26476,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540300,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575220,"statfs":{"total":289910292480,"available":288333025280,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1945,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.500294"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29329,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:01.199757+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:24:01.199757+0000","last_peered":"2024-09-05T13:24:01.199757+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:24:01.199757+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21313,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205482+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.205482+0000","last_peered":"2024-09-05T13:23:41.205482+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.205482+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29674,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202039+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.202039+0000","last_peered":"2024-09-05T13:23:41.202039+0000","last_clean":"2024-09-05T13:23:41.202039+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.202039+0000","last_undegraded":"2024-09-05T13:23:41.202039+0000","last_fullsized":"2024-09-05T13:23:41.202039+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21038,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.206015+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.206015+0000","last_peered":"2024-09-05T13:23:41.206015+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.206015+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26475,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491708,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880132,"statfs":{"total":96636764160,"available":96133255168,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:02.727 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:03.055 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:03.068 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-05T13:24:02.261910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26476,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540300,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575220,"statfs":{"total":289910292480,"available":288333025280,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1945,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.500294"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29329,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:01.199757+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:24:01.199757+0000","last_peered":"2024-09-05T13:24:01.199757+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:24:01.199757+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21313,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205482+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.205482+0000","last_peered":"2024-09-05T13:23:41.205482+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.205482+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29674,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202039+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.202039+0000","last_peered":"2024-09-05T13:23:41.202039+0000","last_clean":"2024-09-05T13:23:41.202039+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.202039+0000","last_undegraded":"2024-09-05T13:23:41.202039+0000","last_fullsized":"2024-09-05T13:23:41.202039+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21038,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.206015+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.206015+0000","last_peered":"2024-09-05T13:23:41.206015+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.206015+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26475,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491708,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880132,"statfs":{"total":96636764160,"available":96133255168,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:03.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-05T13:24:03.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'20972", 'reported_seq': 29329, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:01.199757+0000', 'last_change': '2024-09-05T13:23:39.452618+0000', 'last_active': '2024-09-05T13:24:01.199757+0000', 'last_peered': '2024-09-05T13:24:01.199757+0000', 'last_clean': '2024-09-05T13:13:15.910891+0000', 'last_became_active': '2024-09-05T13:13:18.485219+0000', 'last_became_peered': '2024-09-05T13:13:18.485219+0000', 'last_unstale': '2024-09-05T13:24:01.199757+0000', 'last_undegraded': '2024-09-05T13:13:17.994522+0000', 'last_fullsized': '2024-09-05T13:13:17.904120+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 290, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20972, 'num_write_kb': 41944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4131, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -290}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:03.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:03.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21313, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.205482+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:23:41.205482+0000', 'last_peered': '2024-09-05T13:23:41.205482+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:23:41.205482+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21067, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204706+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:23:41.204706+0000', 'last_peered': '2024-09-05T13:23:41.204706+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:23:41.204706+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 20724, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.204319+0000', 'last_change': '2024-09-05T13:13:18.549605+0000', 'last_active': '2024-09-05T13:23:41.204319+0000', 'last_peered': '2024-09-05T13:23:41.204319+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:23:41.204319+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4389, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4387}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21038, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.206015+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:23:41.206015+0000', 'last_peered': '2024-09-05T13:23:41.206015+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:23:41.206015+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21311, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.205116+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:23:41.205116+0000', 'last_peered': '2024-09-05T13:23:41.205116+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:23:41.205116+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:03.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21351, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:23:41.203415+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:23:41.203415+0000', 'last_peered': '2024-09-05T13:23:41.203415+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:23:41.203415+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:03.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:03.478 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:03.478 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27023/6 objects degraded (450383.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":770,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1577267200,"bytes_avail":288333025280,"bytes_total":289910292480,"degraded_objects":27023,"degraded_total":6,"degraded_ratio":4503.833333333333,"recovering_objects_per_sec":245,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2944,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:03.492 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:03.493 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:03.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:03.841 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:03.856 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-05T13:24:03.763557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26476,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540300,"kb_used_data":215244,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575220,"statfs":{"total":289910292480,"available":288333025280,"internally_reserved":0,"allocated":220409856,"data_stored":219462038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1945,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001327"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29329,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:01.199757+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:24:01.199757+0000","last_peered":"2024-09-05T13:24:01.199757+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:24:01.199757+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20878,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207423+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:23:41.207423+0000","last_peered":"2024-09-05T13:23:41.207423+0000","last_clean":"2024-09-05T13:23:41.207423+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:23:41.207423+0000","last_undegraded":"2024-09-05T13:23:41.207423+0000","last_fullsized":"2024-09-05T13:23:41.207423+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28593,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202106+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:23:41.202106+0000","last_peered":"2024-09-05T13:23:41.202106+0000","last_clean":"2024-09-05T13:23:41.202106+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:23:41.202106+0000","last_undegraded":"2024-09-05T13:23:41.202106+0000","last_fullsized":"2024-09-05T13:23:41.202106+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21115,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204219+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:23:41.204219+0000","last_peered":"2024-09-05T13:23:41.204219+0000","last_clean":"2024-09-05T13:23:41.204219+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:23:41.204219+0000","last_undegraded":"2024-09-05T13:23:41.204219+0000","last_fullsized":"2024-09-05T13:23:41.204219+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21313,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205482+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.205482+0000","last_peered":"2024-09-05T13:23:41.205482+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.205482+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29674,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202039+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:23:41.202039+0000","last_peered":"2024-09-05T13:23:41.202039+0000","last_clean":"2024-09-05T13:23:41.202039+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:23:41.202039+0000","last_undegraded":"2024-09-05T13:23:41.202039+0000","last_fullsized":"2024-09-05T13:23:41.202039+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.201903+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:23:41.201903+0000","last_peered":"2024-09-05T13:23:41.201903+0000","last_clean":"2024-09-05T13:23:41.201903+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:23:41.201903+0000","last_undegraded":"2024-09-05T13:23:41.201903+0000","last_fullsized":"2024-09-05T13:23:41.201903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21246,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.325696+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:23:41.325696+0000","last_peered":"2024-09-05T13:23:41.325696+0000","last_clean":"2024-09-05T13:23:41.325696+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:23:41.325696+0000","last_undegraded":"2024-09-05T13:23:41.325696+0000","last_fullsized":"2024-09-05T13:23:41.325696+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203899+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:23:41.203899+0000","last_peered":"2024-09-05T13:23:41.203899+0000","last_clean":"2024-09-05T13:23:41.203899+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:23:41.203899+0000","last_undegraded":"2024-09-05T13:23:41.203899+0000","last_fullsized":"2024-09-05T13:23:41.203899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29831,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203976+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:23:41.203976+0000","last_peered":"2024-09-05T13:23:41.203976+0000","last_clean":"2024-09-05T13:23:41.203976+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:23:41.203976+0000","last_undegraded":"2024-09-05T13:23:41.203976+0000","last_fullsized":"2024-09-05T13:23:41.203976+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202053+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:23:41.202053+0000","last_peered":"2024-09-05T13:23:41.202053+0000","last_clean":"2024-09-05T13:23:41.202053+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:23:41.202053+0000","last_undegraded":"2024-09-05T13:23:41.202053+0000","last_fullsized":"2024-09-05T13:23:41.202053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":20724,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204319+0000","last_change":"2024-09-05T13:13:18.549605+0000","last_active":"2024-09-05T13:23:41.204319+0000","last_peered":"2024-09-05T13:23:41.204319+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:23:41.204319+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4389,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21038,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.206015+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.206015+0000","last_peered":"2024-09-05T13:23:41.206015+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.206015+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202140+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:23:41.202140+0000","last_peered":"2024-09-05T13:23:41.202140+0000","last_clean":"2024-09-05T13:23:41.202140+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:23:41.202140+0000","last_undegraded":"2024-09-05T13:23:41.202140+0000","last_fullsized":"2024-09-05T13:23:41.202140+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.207155+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:23:41.207155+0000","last_peered":"2024-09-05T13:23:41.207155+0000","last_clean":"2024-09-05T13:23:41.207155+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:23:41.207155+0000","last_undegraded":"2024-09-05T13:23:41.207155+0000","last_fullsized":"2024-09-05T13:23:41.207155+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21243,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202193+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:23:41.202193+0000","last_peered":"2024-09-05T13:23:41.202193+0000","last_clean":"2024-09-05T13:23:41.202193+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:23:41.202193+0000","last_undegraded":"2024-09-05T13:23:41.202193+0000","last_fullsized":"2024-09-05T13:23:41.202193+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202441+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:23:41.202441+0000","last_peered":"2024-09-05T13:23:41.202441+0000","last_clean":"2024-09-05T13:23:41.202441+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:23:41.202441+0000","last_undegraded":"2024-09-05T13:23:41.202441+0000","last_fullsized":"2024-09-05T13:23:41.202441+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.202364+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:23:41.202364+0000","last_peered":"2024-09-05T13:23:41.202364+0000","last_clean":"2024-09-05T13:23:41.202364+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:23:41.202364+0000","last_undegraded":"2024-09-05T13:23:41.202364+0000","last_fullsized":"2024-09-05T13:23:41.202364+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26475,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219074560,"data_stored":219016752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486132,"kb_used_data":143452,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885708,"statfs":{"total":96636764160,"available":96138964992,"internally_reserved":0,"allocated":146894848,"data_stored":146597283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491708,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880132,"statfs":{"total":96636764160,"available":96133255168,"internally_reserved":0,"allocated":503808,"data_stored":155197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:06.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:07.318 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:07.332 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-05T13:24:05.763875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26797,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532004,"kb_used_data":215268,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583516,"statfs":{"total":289910292480,"available":288341520384,"internally_reserved":0,"allocated":220434432,"data_stored":219472550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1450,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001302"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29329,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:01.199757+0000","last_change":"2024-09-05T13:23:39.452618+0000","last_active":"2024-09-05T13:24:01.199757+0000","last_peered":"2024-09-05T13:24:01.199757+0000","last_clean":"2024-09-05T13:13:15.910891+0000","last_became_active":"2024-09-05T13:13:18.485219+0000","last_became_peered":"2024-09-05T13:13:18.485219+0000","last_unstale":"2024-09-05T13:24:01.199757+0000","last_undegraded":"2024-09-05T13:13:17.994522+0000","last_fullsized":"2024-09-05T13:13:17.904120+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21036,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204023+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:23:41.204023+0000","last_peered":"2024-09-05T13:23:41.204023+0000","last_clean":"2024-09-05T13:23:41.204023+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:23:41.204023+0000","last_undegraded":"2024-09-05T13:23:41.204023+0000","last_fullsized":"2024-09-05T13:23:41.204023+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21313,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205482+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:23:41.205482+0000","last_peered":"2024-09-05T13:23:41.205482+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:23:41.205482+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21255,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.203024+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:23:41.203024+0000","last_peered":"2024-09-05T13:23:41.203024+0000","last_clean":"2024-09-05T13:23:41.203024+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:23:41.203024+0000","last_undegraded":"2024-09-05T13:23:41.203024+0000","last_fullsized":"2024-09-05T13:23:41.203024+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21067,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.204706+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:23:41.204706+0000","last_peered":"2024-09-05T13:23:41.204706+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:23:41.204706+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":123,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-05T13:23:41.204615+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:23:41.204615+0000","last_peered":"2024-09-05T13:23:41.204615+0000","last_clean":"2024-09-05T13:23:41.204615+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:23:41.204615+0000","last_undegraded":"2024-09-05T13:23:41.204615+0000","last_fullsized":"2024-09-05T13:23:41.204615+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":21371,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:04.131632+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:04.131632+0000","last_peered":"2024-09-05T13:24:04.131632+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:04.131632+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21038,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.206015+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:23:41.206015+0000","last_peered":"2024-09-05T13:23:41.206015+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:23:41.206015+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.205116+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:23:41.205116+0000","last_peered":"2024-09-05T13:23:41.205116+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:23:41.205116+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260209+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260209+0000","last_peered":"2024-09-05T13:24:03.260209+0000","last_clean":"2024-09-05T13:24:03.260209+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260209+0000","last_undegraded":"2024-09-05T13:24:03.260209+0000","last_fullsized":"2024-09-05T13:24:03.260209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21351,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:23:41.203415+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:23:41.203415+0000","last_peered":"2024-09-05T13:23:41.203415+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:23:41.203415+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26796,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562460,"kb_used_data":71300,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809380,"statfs":{"total":96636764160,"available":96060805120,"internally_reserved":0,"allocated":73011200,"data_stored":72709558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483396,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888444,"statfs":{"total":96636764160,"available":96141766656,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:07.333 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:07.777 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:07.790 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-05T13:24:07.764302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27087,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532020,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583500,"statfs":{"total":289910292480,"available":288341504000,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1740,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001428"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":21371,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:04.131632+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:04.131632+0000","last_peered":"2024-09-05T13:24:04.131632+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:04.131632+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260209+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260209+0000","last_peered":"2024-09-05T13:24:03.260209+0000","last_clean":"2024-09-05T13:24:03.260209+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260209+0000","last_undegraded":"2024-09-05T13:24:03.260209+0000","last_fullsized":"2024-09-05T13:24:03.260209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27086,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483396,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888444,"statfs":{"total":96636764160,"available":96141766656,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:07.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21317, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260793+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:03.260793+0000', 'last_peered': '2024-09-05T13:24:03.260793+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:03.260793+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21071, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260882+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:24:03.260882+0000', 'last_peered': '2024-09-05T13:24:03.260882+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:03.260882+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 21371, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:04.131632+0000', 'last_change': '2024-09-05T13:24:02.099328+0000', 'last_active': '2024-09-05T13:24:04.131632+0000', 'last_peered': '2024-09-05T13:24:04.131632+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:24:04.131632+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4068, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 321, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4068}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21042, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260708+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:03.260708+0000', 'last_peered': '2024-09-05T13:24:03.260708+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:03.260708+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:07.792 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21315, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.261093+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:03.261093+0000', 'last_peered': '2024-09-05T13:24:03.261093+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:03.261093+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:07.793 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:07.793 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21355, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260836+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:03.260836+0000', 'last_peered': '2024-09-05T13:24:03.260836+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:03.260836+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:07.793 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:08.281 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:08.281 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26412/6 objects degraded (440200.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":775,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1568788480,"bytes_avail":288341504000,"bytes_total":289910292480,"degraded_objects":26412,"degraded_total":6,"degraded_ratio":4402,"recovering_objects_per_sec":217,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1740,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:08.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:08.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:08.668 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:08.683 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-05T13:24:07.764302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27087,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1532020,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281583500,"statfs":{"total":289910292480,"available":288341504000,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1348258822},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1740,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001428"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":21371,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:04.131632+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:04.131632+0000","last_peered":"2024-09-05T13:24:04.131632+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:04.131632+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260209+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260209+0000","last_peered":"2024-09-05T13:24:03.260209+0000","last_clean":"2024-09-05T13:24:03.260209+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260209+0000","last_undegraded":"2024-09-05T13:24:03.260209+0000","last_fullsized":"2024-09-05T13:24:03.260209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27086,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483396,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888444,"statfs":{"total":96636764160,"available":96141766656,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:11.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:12.042 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:12.043 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:12.055 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-05T13:24:11.764993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28085,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540340,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575180,"statfs":{"total":289910292480,"available":288332984320,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.503082"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":23367,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:09.137830+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:09.137830+0000","last_peered":"2024-09-05T13:24:09.137830+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:09.137830+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1319,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28084,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491716,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880124,"statfs":{"total":96636764160,"available":96133246976,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:12.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:12.434 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:12.435 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:12.448 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-05T13:24:11.764993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28085,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540340,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575180,"statfs":{"total":289910292480,"available":288332984320,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.503082"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":23367,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:09.137830+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:09.137830+0000","last_peered":"2024-09-05T13:24:09.137830+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:09.137830+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1319,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28084,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491716,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880124,"statfs":{"total":96636764160,"available":96133246976,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21317, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260793+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:03.260793+0000', 'last_peered': '2024-09-05T13:24:03.260793+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:03.260793+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21071, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260882+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:24:03.260882+0000', 'last_peered': '2024-09-05T13:24:03.260882+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:03.260882+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 23367, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:09.137830+0000', 'last_change': '2024-09-05T13:24:02.099328+0000', 'last_active': '2024-09-05T13:24:09.137830+0000', 'last_peered': '2024-09-05T13:24:09.137830+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:24:09.137830+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1319, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3070}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21042, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260708+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:03.260708+0000', 'last_peered': '2024-09-05T13:24:03.260708+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:03.260708+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21315, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.261093+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:03.261093+0000', 'last_peered': '2024-09-05T13:24:03.261093+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:03.261093+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:12.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:12.451 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21355, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260836+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:03.260836+0000', 'last_peered': '2024-09-05T13:24:03.260836+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:03.260836+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:12.451 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:12.908 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:12.908 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25414/6 objects degraded (423566.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":779,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1577308160,"bytes_avail":288332984320,"bytes_total":289910292480,"degraded_objects":25414,"degraded_total":6,"degraded_ratio":4235.666666666667,"recovering_objects_per_sec":169,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:12.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:12.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:13.243 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:13.244 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:13.256 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-05T13:24:11.764993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28085,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540340,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1324979,"kb_avail":281575180,"statfs":{"total":289910292480,"available":288332984320,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1356778502},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1609,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1609,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.503082"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":23367,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:09.137830+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:09.137830+0000","last_peered":"2024-09-05T13:24:09.137830+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:09.137830+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1319,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28084,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491716,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880124,"statfs":{"total":96636764160,"available":96133246976,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:16.258 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:16.582 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:16.582 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:16.595 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-05T13:24:15.765700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29272,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1550068,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281565452,"statfs":{"total":289910292480,"available":288323022848,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1366739974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002142"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":25741,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:14.138172+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:14.138172+0000","last_peered":"2024-09-05T13:24:14.138172+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:14.138172+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1883}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29271,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:16.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:16.924 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:16.938 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-05T13:24:15.765700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29272,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1550068,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281565452,"statfs":{"total":289910292480,"available":288323022848,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1366739974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002142"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":25741,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:14.138172+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:14.138172+0000","last_peered":"2024-09-05T13:24:14.138172+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:14.138172+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1883}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29271,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:16.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:16.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21317, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260793+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:03.260793+0000', 'last_peered': '2024-09-05T13:24:03.260793+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:03.260793+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:16.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:16.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21071, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260882+0000', 'last_change': '2024-09-05T13:13:18.545428+0000', 'last_active': '2024-09-05T13:24:03.260882+0000', 'last_peered': '2024-09-05T13:24:03.260882+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:03.260882+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4456, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:16.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:24:16.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 25741, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:14.138172+0000', 'last_change': '2024-09-05T13:24:02.099328+0000', 'last_active': '2024-09-05T13:24:14.138172+0000', 'last_peered': '2024-09-05T13:24:14.138172+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:24:14.138172+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1883, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2506, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1883}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:24:16.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:16.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21042, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260708+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:03.260708+0000', 'last_peered': '2024-09-05T13:24:03.260708+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:03.260708+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:16.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:16.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21315, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.261093+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:03.261093+0000', 'last_peered': '2024-09-05T13:24:03.261093+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:03.261093+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:16.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:16.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21355, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260836+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:03.260836+0000', 'last_peered': '2024-09-05T13:24:03.260836+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:03.260836+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:16.940 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:17.360 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:17.360 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24227/6 objects degraded (403783.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":784,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1587269632,"bytes_avail":288323022848,"bytes_total":289910292480,"degraded_objects":24227,"degraded_total":6,"degraded_ratio":4037.8333333333339,"recovering_objects_per_sec":232,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2796,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:17.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:17.374 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:17.690 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:17.691 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:17.702 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-05T13:24:15.765700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29272,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1550068,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281565452,"statfs":{"total":289910292480,"available":288323022848,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1366739974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002142"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":25741,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:14.138172+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:14.138172+0000","last_peered":"2024-09-05T13:24:14.138172+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:14.138172+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2506,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1883}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29271,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:20.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:21.070 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:21.070 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:21.083 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-05T13:24:19.766443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30717,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1550068,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281565452,"statfs":{"total":289910292480,"available":288323022848,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1366739974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3630,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3630,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002140"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21071,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260882+0000","last_change":"2024-09-05T13:13:18.545428+0000","last_active":"2024-09-05T13:24:03.260882+0000","last_peered":"2024-09-05T13:24:03.260882+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:03.260882+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":28632,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:19.140195+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:19.140195+0000","last_peered":"2024-09-05T13:24:19.140195+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:19.140195+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-438}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22782,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30716,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219066368,"data_stored":219015786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:21.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:21.398 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:21.410 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-05T13:24:21.314033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30773,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545908,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281569612,"statfs":{"total":289910292480,"available":288327282688,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1362480134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2688,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.549396"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21184,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:21.193615+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:21.193615+0000","last_peered":"2024-09-05T13:24:21.193615+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:21.193615+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4400,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4400}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":28632,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:19.140195+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:19.140195+0000","last_peered":"2024-09-05T13:24:19.140195+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:19.140195+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-438}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30772,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219058176,"data_stored":219014820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:21.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:21.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21317, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260793+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:03.260793+0000', 'last_peered': '2024-09-05T13:24:03.260793+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:03.260793+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:21.411 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21184, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:21.193615+0000', 'last_change': '2024-09-05T13:24:20.536499+0000', 'last_active': '2024-09-05T13:24:21.193615+0000', 'last_peered': '2024-09-05T13:24:21.193615+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:21.193615+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4400, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 57, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4400}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "47'20582", 'reported_seq': 28632, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:19.140195+0000', 'last_change': '2024-09-05T13:24:02.099328+0000', 'last_active': '2024-09-05T13:24:19.140195+0000', 'last_peered': '2024-09-05T13:24:19.140195+0000', 'last_clean': '2024-09-05T13:13:15.890176+0000', 'last_became_active': '2024-09-05T13:13:18.534177+0000', 'last_became_peered': '2024-09-05T13:13:18.534177+0000', 'last_unstale': '2024-09-05T13:24:19.140195+0000', 'last_undegraded': '2024-09-05T13:13:17.942632+0000', 'last_fullsized': '2024-09-05T13:13:17.903545+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2537", 'last_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:51.179242+0000', 'objects_scrubbed': 2485, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1019, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18091, 'num_write_kb': 31212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3951, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -438}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21042, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260708+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:03.260708+0000', 'last_peered': '2024-09-05T13:24:03.260708+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:03.260708+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21315, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.261093+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:03.261093+0000', 'last_peered': '2024-09-05T13:24:03.261093+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:03.261093+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:21.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21355, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260836+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:03.260836+0000', 'last_peered': '2024-09-05T13:24:03.260836+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:03.260836+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:21.413 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:21.790 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22782/6 objects degraded (379700.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":788,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1587269632,"bytes_avail":288323022848,"bytes_total":289910292480,"degraded_objects":22782,"degraded_total":6,"degraded_ratio":3797,"recovering_objects_per_sec":302,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":3630,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:21.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:21.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:22.141 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:22.156 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-05T13:24:21.766766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30773,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545908,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281569612,"statfs":{"total":289910292480,"available":288327282688,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1362480134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2688,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001772"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21184,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:21.193615+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:21.193615+0000","last_peered":"2024-09-05T13:24:21.193615+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:21.193615+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4400,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4400}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":28632,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:19.140195+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:19.140195+0000","last_peered":"2024-09-05T13:24:19.140195+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:19.140195+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-438}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30772,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219058176,"data_stored":219014820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:25.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:25.509 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:25.509 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:25.523 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-05T13:24:23.767157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30773,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545908,"kb_used_data":215284,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281569612,"statfs":{"total":289910292480,"available":288327282688,"internally_reserved":0,"allocated":220450816,"data_stored":219478289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1362480134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001456"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20882,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.265047+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:03.265047+0000","last_peered":"2024-09-05T13:24:03.265047+0000","last_clean":"2024-09-05T13:24:03.265047+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:03.265047+0000","last_undegraded":"2024-09-05T13:24:03.265047+0000","last_fullsized":"2024-09-05T13:24:03.265047+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28597,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262980+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:03.262980+0000","last_peered":"2024-09-05T13:24:03.262980+0000","last_clean":"2024-09-05T13:24:03.262980+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:03.262980+0000","last_undegraded":"2024-09-05T13:24:03.262980+0000","last_fullsized":"2024-09-05T13:24:03.262980+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21119,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264664+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:03.264664+0000","last_peered":"2024-09-05T13:24:03.264664+0000","last_clean":"2024-09-05T13:24:03.264664+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:03.264664+0000","last_undegraded":"2024-09-05T13:24:03.264664+0000","last_fullsized":"2024-09-05T13:24:03.264664+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29678,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261687+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:03.261687+0000","last_peered":"2024-09-05T13:24:03.261687+0000","last_clean":"2024-09-05T13:24:03.261687+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:03.261687+0000","last_undegraded":"2024-09-05T13:24:03.261687+0000","last_fullsized":"2024-09-05T13:24:03.261687+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260300+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:03.260300+0000","last_peered":"2024-09-05T13:24:03.260300+0000","last_clean":"2024-09-05T13:24:03.260300+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:03.260300+0000","last_undegraded":"2024-09-05T13:24:03.260300+0000","last_fullsized":"2024-09-05T13:24:03.260300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21250,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262146+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:03.262146+0000","last_peered":"2024-09-05T13:24:03.262146+0000","last_clean":"2024-09-05T13:24:03.262146+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:03.262146+0000","last_undegraded":"2024-09-05T13:24:03.262146+0000","last_fullsized":"2024-09-05T13:24:03.262146+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264456+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:03.264456+0000","last_peered":"2024-09-05T13:24:03.264456+0000","last_clean":"2024-09-05T13:24:03.264456+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:03.264456+0000","last_undegraded":"2024-09-05T13:24:03.264456+0000","last_fullsized":"2024-09-05T13:24:03.264456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21184,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:21.193615+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:21.193615+0000","last_peered":"2024-09-05T13:24:21.193615+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:21.193615+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4400,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4400}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29835,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264482+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:03.264482+0000","last_peered":"2024-09-05T13:24:03.264482+0000","last_clean":"2024-09-05T13:24:03.264482+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:03.264482+0000","last_undegraded":"2024-09-05T13:24:03.264482+0000","last_fullsized":"2024-09-05T13:24:03.264482+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262212+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:03.262212+0000","last_peered":"2024-09-05T13:24:03.262212+0000","last_clean":"2024-09-05T13:24:03.262212+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:03.262212+0000","last_undegraded":"2024-09-05T13:24:03.262212+0000","last_fullsized":"2024-09-05T13:24:03.262212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":28632,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:19.140195+0000","last_change":"2024-09-05T13:24:02.099328+0000","last_active":"2024-09-05T13:24:19.140195+0000","last_peered":"2024-09-05T13:24:19.140195+0000","last_clean":"2024-09-05T13:13:15.890176+0000","last_became_active":"2024-09-05T13:13:18.534177+0000","last_became_peered":"2024-09-05T13:13:18.534177+0000","last_unstale":"2024-09-05T13:24:19.140195+0000","last_undegraded":"2024-09-05T13:13:17.942632+0000","last_fullsized":"2024-09-05T13:13:17.903545+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-438}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.262588+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:03.262588+0000","last_peered":"2024-09-05T13:24:03.262588+0000","last_clean":"2024-09-05T13:24:03.262588+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:03.262588+0000","last_undegraded":"2024-09-05T13:24:03.262588+0000","last_fullsized":"2024-09-05T13:24:03.262588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.264752+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:03.264752+0000","last_peered":"2024-09-05T13:24:03.264752+0000","last_clean":"2024-09-05T13:24:03.264752+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:03.264752+0000","last_undegraded":"2024-09-05T13:24:03.264752+0000","last_fullsized":"2024-09-05T13:24:03.264752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21247,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263005+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:03.263005+0000","last_peered":"2024-09-05T13:24:03.263005+0000","last_clean":"2024-09-05T13:24:03.263005+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:03.263005+0000","last_undegraded":"2024-09-05T13:24:03.263005+0000","last_fullsized":"2024-09-05T13:24:03.263005+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.261103+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:03.261103+0000","last_peered":"2024-09-05T13:24:03.261103+0000","last_clean":"2024-09-05T13:24:03.261103+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:03.261103+0000","last_undegraded":"2024-09-05T13:24:03.261103+0000","last_fullsized":"2024-09-05T13:24:03.261103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260253+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:03.260253+0000","last_peered":"2024-09-05T13:24:03.260253+0000","last_clean":"2024-09-05T13:24:03.260253+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:03.260253+0000","last_undegraded":"2024-09-05T13:24:03.260253+0000","last_fullsized":"2024-09-05T13:24:03.260253+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30772,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219058176,"data_stored":219014820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486148,"kb_used_data":143468,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93885692,"statfs":{"total":96636764160,"available":96138948608,"internally_reserved":0,"allocated":146911232,"data_stored":146603022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497284,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874556,"statfs":{"total":96636764160,"available":96127545344,"internally_reserved":0,"allocated":512000,"data_stored":159970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":146452480,"data_stored":146448867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:25.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:25.853 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:25.854 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:25.868 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-05T13:24:25.767453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31211,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500564,"kb_used_data":161624,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614956,"statfs":{"total":289910292480,"available":288373714944,"internally_reserved":0,"allocated":165502976,"data_stored":164516285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001421"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21184,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:21.193615+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:21.193615+0000","last_peered":"2024-09-05T13:24:21.193615+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:21.193615+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4400,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4400}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31210,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505612,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866228,"statfs":{"total":96636764160,"available":96119017472,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:25.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:25.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21317, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260793+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:03.260793+0000', 'last_peered': '2024-09-05T13:24:03.260793+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:03.260793+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:25.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:25.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 21184, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:21.193615+0000', 'last_change': '2024-09-05T13:24:20.536499+0000', 'last_active': '2024-09-05T13:24:21.193615+0000', 'last_peered': '2024-09-05T13:24:21.193615+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:21.193615+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4400, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 57, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4400}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:25.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:25.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21042, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260708+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:03.260708+0000', 'last_peered': '2024-09-05T13:24:03.260708+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:03.260708+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:25.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:25.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21315, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.261093+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:03.261093+0000', 'last_peered': '2024-09-05T13:24:03.261093+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:03.261093+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:25.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:25.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21355, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:03.260836+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:03.260836+0000', 'last_peered': '2024-09-05T13:24:03.260836+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:03.260836+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:25.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:26.303 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:26.303 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22726/6 objects degraded (378766.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":793,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1583009792,"bytes_avail":288327282688,"bytes_total":289910292480,"degraded_objects":22726,"degraded_total":6,"degraded_ratio":3787.6666666666661,"recovering_objects_per_sec":187,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1501,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:26.316 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:26.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:26.671 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:26.671 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:26.683 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-05T13:24:25.767453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31211,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500564,"kb_used_data":161624,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614956,"statfs":{"total":289910292480,"available":288373714944,"internally_reserved":0,"allocated":165502976,"data_stored":164516285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001421"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29920,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.329485+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:03.329485+0000","last_peered":"2024-09-05T13:24:03.329485+0000","last_clean":"2024-09-05T13:24:03.329485+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:03.329485+0000","last_undegraded":"2024-09-05T13:24:03.329485+0000","last_fullsized":"2024-09-05T13:24:03.329485+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21041,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260954+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:03.260954+0000","last_peered":"2024-09-05T13:24:03.260954+0000","last_clean":"2024-09-05T13:24:03.260954+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:03.260954+0000","last_undegraded":"2024-09-05T13:24:03.260954+0000","last_fullsized":"2024-09-05T13:24:03.260954+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21317,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260793+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:03.260793+0000","last_peered":"2024-09-05T13:24:03.260793+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:03.260793+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21259,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.263673+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:03.263673+0000","last_peered":"2024-09-05T13:24:03.263673+0000","last_clean":"2024-09-05T13:24:03.263673+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:03.263673+0000","last_undegraded":"2024-09-05T13:24:03.263673+0000","last_fullsized":"2024-09-05T13:24:03.263673+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":21184,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:21.193615+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:21.193615+0000","last_peered":"2024-09-05T13:24:21.193615+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:21.193615+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4400,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4400}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-05T13:24:03.260688+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:03.260688+0000","last_peered":"2024-09-05T13:24:03.260688+0000","last_clean":"2024-09-05T13:24:03.260688+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:03.260688+0000","last_undegraded":"2024-09-05T13:24:03.260688+0000","last_fullsized":"2024-09-05T13:24:03.260688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21042,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260708+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:03.260708+0000","last_peered":"2024-09-05T13:24:03.260708+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:03.260708+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.261093+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:03.261093+0000","last_peered":"2024-09-05T13:24:03.261093+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:03.261093+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21355,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:03.260836+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:03.260836+0000","last_peered":"2024-09-05T13:24:03.260836+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:03.260836+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31210,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562476,"kb_used_data":71316,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809364,"statfs":{"total":96636764160,"available":96060788736,"internally_reserved":0,"allocated":73027584,"data_stored":72715297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505612,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866228,"statfs":{"total":96636764160,"available":96119017472,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:29.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:29.998 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:29.999 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:30.010 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-05T13:24:29.768220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32067,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500576,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614944,"statfs":{"total":289910292480,"available":288373702656,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1294,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.454186"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":22900,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:26.207735+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:26.207735+0000","last_peered":"2024-09-05T13:24:26.207735+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:26.207735+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":913,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323274+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323274+0000","last_peered":"2024-09-05T13:24:22.323274+0000","last_clean":"2024-09-05T13:24:22.323274+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323274+0000","last_undegraded":"2024-09-05T13:24:22.323274+0000","last_fullsized":"2024-09-05T13:24:22.323274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32066,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505612,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866228,"statfs":{"total":96636764160,"available":96119017472,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:30.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:30.357 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:30.370 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-05T13:24:29.768220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32067,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500576,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614944,"statfs":{"total":289910292480,"available":288373702656,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1294,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.454186"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":22900,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:26.207735+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:26.207735+0000","last_peered":"2024-09-05T13:24:26.207735+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:26.207735+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":913,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323274+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323274+0000","last_peered":"2024-09-05T13:24:22.323274+0000","last_clean":"2024-09-05T13:24:22.323274+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323274+0000","last_undegraded":"2024-09-05T13:24:22.323274+0000","last_fullsized":"2024-09-05T13:24:22.323274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32066,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505612,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866228,"statfs":{"total":96636764160,"available":96119017472,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:30.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:30.371 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21321, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.323360+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:22.323360+0000', 'last_peered': '2024-09-05T13:24:22.323360+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:22.323360+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:30.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:30.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 22900, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:26.207735+0000', 'last_change': '2024-09-05T13:24:20.536499+0000', 'last_active': '2024-09-05T13:24:26.207735+0000', 'last_peered': '2024-09-05T13:24:26.207735+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:26.207735+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3544, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 913, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3544}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:30.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:30.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21046, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322680+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:22.322680+0000', 'last_peered': '2024-09-05T13:24:22.322680+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:22.322680+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:30.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:30.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21319, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322660+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:22.322660+0000', 'last_peered': '2024-09-05T13:24:22.322660+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:22.322660+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:30.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:30.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21359, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322815+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:22.322815+0000', 'last_peered': '2024-09-05T13:24:22.322815+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:22.322815+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:30.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:30.760 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:30.761 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21432/6 objects degraded (357200.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":797,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536589824,"bytes_avail":288373702656,"bytes_total":289910292480,"degraded_objects":21432,"degraded_total":6,"degraded_ratio":3572,"recovering_objects_per_sec":153,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1294,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:30.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:30.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:31.067 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:31.067 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:31.079 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-09-05T13:24:29.768220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32067,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500576,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614944,"statfs":{"total":289910292480,"available":288373702656,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1294,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.454186"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":22900,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:26.207735+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:26.207735+0000","last_peered":"2024-09-05T13:24:26.207735+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:26.207735+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3544,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":913,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3544}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323274+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323274+0000","last_peered":"2024-09-05T13:24:22.323274+0000","last_clean":"2024-09-05T13:24:22.323274+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323274+0000","last_undegraded":"2024-09-05T13:24:22.323274+0000","last_fullsized":"2024-09-05T13:24:22.323274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32066,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505612,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866228,"statfs":{"total":96636764160,"available":96119017472,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:34.082 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:34.418 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:34.418 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:34.431 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-05T13:24:33.768869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32657,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1504736,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281610784,"statfs":{"total":289910292480,"available":288369442816,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1375259654},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.448530"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":24080,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:31.206060+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:31.206060+0000","last_peered":"2024-09-05T13:24:31.206060+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:31.206060+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1503,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2954}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323274+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323274+0000","last_peered":"2024-09-05T13:24:22.323274+0000","last_clean":"2024-09-05T13:24:22.323274+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323274+0000","last_undegraded":"2024-09-05T13:24:22.323274+0000","last_fullsized":"2024-09-05T13:24:22.323274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32656,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:34.432 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:34.734 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:34.734 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:34.746 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-05T13:24:33.768869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32657,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1504736,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281610784,"statfs":{"total":289910292480,"available":288369442816,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1375259654},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.448530"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":24080,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:31.206060+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:31.206060+0000","last_peered":"2024-09-05T13:24:31.206060+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:31.206060+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1503,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2954}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323274+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323274+0000","last_peered":"2024-09-05T13:24:22.323274+0000","last_clean":"2024-09-05T13:24:22.323274+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323274+0000","last_undegraded":"2024-09-05T13:24:22.323274+0000","last_fullsized":"2024-09-05T13:24:22.323274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32656,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:34.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:34.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21321, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.323360+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:22.323360+0000', 'last_peered': '2024-09-05T13:24:22.323360+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:22.323360+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:34.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:34.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 24080, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:31.206060+0000', 'last_change': '2024-09-05T13:24:20.536499+0000', 'last_active': '2024-09-05T13:24:31.206060+0000', 'last_peered': '2024-09-05T13:24:31.206060+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:31.206060+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2954, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1503, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2954}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:34.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:34.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21046, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322680+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:22.322680+0000', 'last_peered': '2024-09-05T13:24:22.322680+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:22.322680+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:34.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:34.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21319, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322660+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:22.322660+0000', 'last_peered': '2024-09-05T13:24:22.322660+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:22.322660+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:34.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:34.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21359, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322815+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:22.322815+0000', 'last_peered': '2024-09-05T13:24:22.322815+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:22.322815+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:34.749 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:35.185 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:35.185 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20842/6 objects degraded (347366.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":802,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1540849664,"bytes_avail":288369442816,"bytes_total":289910292480,"degraded_objects":20842,"degraded_total":6,"degraded_ratio":3473.6666666666661,"recovering_objects_per_sec":164,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:35.198 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:35.198 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:35.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:35.505 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:35.517 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-05T13:24:33.768869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32657,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1504736,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281610784,"statfs":{"total":289910292480,"available":288369442816,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1375259654},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.448530"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":24080,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:31.206060+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:31.206060+0000","last_peered":"2024-09-05T13:24:31.206060+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:31.206060+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1503,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2954}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323274+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323274+0000","last_peered":"2024-09-05T13:24:22.323274+0000","last_clean":"2024-09-05T13:24:22.323274+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323274+0000","last_undegraded":"2024-09-05T13:24:22.323274+0000","last_fullsized":"2024-09-05T13:24:22.323274+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32656,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:38.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:38.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:38.859 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-05T13:24:37.769471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33300,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496416,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281619104,"statfs":{"total":289910292480,"available":288377962496,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1366739974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2089,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002017"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":25367,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:36.205966+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:36.205966+0000","last_peered":"2024-09-05T13:24:36.205966+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:36.205966+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2311}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33299,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501452,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870388,"statfs":{"total":96636764160,"available":96123277312,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:38.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:39.223 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:39.223 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:39.236 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-05T13:24:37.769471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33300,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496416,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281619104,"statfs":{"total":289910292480,"available":288377962496,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1366739974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2089,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2089,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002017"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":25367,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:36.205966+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:36.205966+0000","last_peered":"2024-09-05T13:24:36.205966+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:36.205966+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2311}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33299,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501452,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870388,"statfs":{"total":96636764160,"available":96123277312,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21321, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.323360+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:22.323360+0000', 'last_peered': '2024-09-05T13:24:22.323360+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:22.323360+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 25367, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:36.205966+0000', 'last_change': '2024-09-05T13:24:20.536499+0000', 'last_active': '2024-09-05T13:24:36.205966+0000', 'last_peered': '2024-09-05T13:24:36.205966+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:36.205966+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2311, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2146, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2311}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21046, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322680+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:22.322680+0000', 'last_peered': '2024-09-05T13:24:22.322680+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:22.322680+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21319, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322660+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:22.322660+0000', 'last_peered': '2024-09-05T13:24:22.322660+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:22.322660+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21359, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322815+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:22.322815+0000', 'last_peered': '2024-09-05T13:24:22.322815+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:22.322815+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:39.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:39.612 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:39.613 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20199/6 objects degraded (336650.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":806,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1532329984,"bytes_avail":288377962496,"bytes_total":289910292480,"degraded_objects":20199,"degraded_total":6,"degraded_ratio":3366.5,"recovering_objects_per_sec":174,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2089,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:39.626 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:39.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:39.951 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:39.952 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:39.965 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-05T13:24:39.769823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33300,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496416,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281619104,"statfs":{"total":289910292480,"available":288377962496,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1366739974},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002009"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":25367,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:36.205966+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:36.205966+0000","last_peered":"2024-09-05T13:24:36.205966+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:36.205966+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2311}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33299,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501452,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870388,"statfs":{"total":96636764160,"available":96123277312,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:42.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:43.297 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-05T13:24:41.770148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34079,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500576,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614944,"statfs":{"total":289910292480,"available":288373702656,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001927"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":26925,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:41.209951+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:41.209951+0000","last_peered":"2024-09-05T13:24:41.209951+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:41.209951+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2925,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1532}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34078,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505612,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866228,"statfs":{"total":96636764160,"available":96119017472,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:43.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:43.601 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:43.602 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:43.615 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-05T13:24:41.770148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34079,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500576,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614944,"statfs":{"total":289910292480,"available":288373702656,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001927"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":26925,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:41.209951+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:41.209951+0000","last_peered":"2024-09-05T13:24:41.209951+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:41.209951+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2925,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1532}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34078,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505612,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866228,"statfs":{"total":96636764160,"available":96119017472,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:43.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:43.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21321, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.323360+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:22.323360+0000', 'last_peered': '2024-09-05T13:24:22.323360+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:22.323360+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:43.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:43.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 26925, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:41.209951+0000', 'last_change': '2024-09-05T13:24:20.536499+0000', 'last_active': '2024-09-05T13:24:41.209951+0000', 'last_peered': '2024-09-05T13:24:41.209951+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:41.209951+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1532, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2925, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1532}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:43.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:43.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21046, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322680+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:22.322680+0000', 'last_peered': '2024-09-05T13:24:22.322680+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:22.322680+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:43.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:43.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21319, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322660+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:22.322660+0000', 'last_peered': '2024-09-05T13:24:22.322660+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:22.322660+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:43.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:43.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21359, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322815+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:22.322815+0000', 'last_peered': '2024-09-05T13:24:22.322815+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:22.322815+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:43.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:43.994 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:43.994 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19420/6 objects degraded (323666.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":810,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536589824,"bytes_avail":288373702656,"bytes_total":289910292480,"degraded_objects":19420,"degraded_total":6,"degraded_ratio":3236.6666666666661,"recovering_objects_per_sec":167,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2012,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:44.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:44.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:44.314 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:44.327 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-05T13:24:43.770513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34079,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500576,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614944,"statfs":{"total":289910292480,"available":288373702656,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1422,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001980"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":26925,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:41.209951+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:41.209951+0000","last_peered":"2024-09-05T13:24:41.209951+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:41.209951+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2925,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1532}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34078,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505612,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866228,"statfs":{"total":96636764160,"available":96119017472,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:47.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:47.646 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:47.647 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:47.658 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-05T13:24:45.770787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34079,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492256,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281623264,"statfs":{"total":289910292480,"available":288382222336,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1362480134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1422,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001917"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":26925,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:41.209951+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:41.209951+0000","last_peered":"2024-09-05T13:24:41.209951+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:41.209951+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1532,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2925,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1532}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21359,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322815+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.322815+0000","last_peered":"2024-09-05T13:24:22.322815+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.322815+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34078,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497292,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874548,"statfs":{"total":96636764160,"available":96127537152,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:47.660 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:48.022 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:48.022 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:48.034 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-05T13:24:47.771096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35169,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492256,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281623264,"statfs":{"total":289910292480,"available":288382222336,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1362480134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2512,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2512,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001938"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":29105,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:46.209158+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:46.209158+0000","last_peered":"2024-09-05T13:24:46.209158+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:46.209158+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4015,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323959+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.323959+0000","last_peered":"2024-09-05T13:24:22.323959+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.323959+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35168,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497292,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874548,"statfs":{"total":96636764160,"available":96127537152,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21321, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.323360+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:22.323360+0000', 'last_peered': '2024-09-05T13:24:22.323360+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:22.323360+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-05T13:24:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'20910", 'reported_seq': 29105, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:46.209158+0000', 'last_change': '2024-09-05T13:24:20.536499+0000', 'last_active': '2024-09-05T13:24:46.209158+0000', 'last_peered': '2024-09-05T13:24:46.209158+0000', 'last_clean': '2024-09-05T13:13:15.889399+0000', 'last_became_active': '2024-09-05T13:13:18.533106+0000', 'last_became_peered': '2024-09-05T13:13:18.533106+0000', 'last_unstale': '2024-09-05T13:24:46.209158+0000', 'last_undegraded': '2024-09-05T13:13:17.995842+0000', 'last_fullsized': '2024-09-05T13:13:17.904290+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3709", 'last_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:04.159129+0000', 'objects_scrubbed': 3625, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2037, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 442, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17277, 'num_write_kb': 27304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4015, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -442}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21046, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322680+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:22.322680+0000', 'last_peered': '2024-09-05T13:24:22.322680+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:22.322680+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:48.036 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21319, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.322660+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:22.322660+0000', 'last_peered': '2024-09-05T13:24:22.322660+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:22.322660+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:48.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:48.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 21360, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:22.323959+0000', 'last_change': '2024-09-05T13:13:18.552474+0000', 'last_active': '2024-09-05T13:24:22.323959+0000', 'last_peered': '2024-09-05T13:24:22.323959+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:22.323959+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4568, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:48.037 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:48.431 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:48.432 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18330/6 objects degraded (305500.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":815,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1528070144,"bytes_avail":288382222336,"bytes_total":289910292480,"degraded_objects":18330,"degraded_total":6,"degraded_ratio":3055,"recovering_objects_per_sec":209,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2512,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:48.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:48.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:48.783 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:48.800 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-09-05T13:24:48.380494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35169,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1492256,"kb_used_data":161640,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281623264,"statfs":{"total":289910292480,"available":288382222336,"internally_reserved":0,"allocated":165519360,"data_stored":164521974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1362480134},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1869,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1869,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.611022"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29924,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322719+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:22.322719+0000","last_peered":"2024-09-05T13:24:22.322719+0000","last_clean":"2024-09-05T13:24:22.322719+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:22.322719+0000","last_undegraded":"2024-09-05T13:24:22.322719+0000","last_fullsized":"2024-09-05T13:24:22.322719+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20886,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28601,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321768+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:22.321768+0000","last_peered":"2024-09-05T13:24:22.321768+0000","last_clean":"2024-09-05T13:24:22.321768+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:22.321768+0000","last_undegraded":"2024-09-05T13:24:22.321768+0000","last_fullsized":"2024-09-05T13:24:22.321768+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21123,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323726+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:22.323726+0000","last_peered":"2024-09-05T13:24:22.323726+0000","last_clean":"2024-09-05T13:24:22.323726+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:22.323726+0000","last_undegraded":"2024-09-05T13:24:22.323726+0000","last_fullsized":"2024-09-05T13:24:22.323726+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21045,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.322518+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:22.322518+0000","last_peered":"2024-09-05T13:24:22.322518+0000","last_clean":"2024-09-05T13:24:22.322518+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:22.322518+0000","last_undegraded":"2024-09-05T13:24:22.322518+0000","last_fullsized":"2024-09-05T13:24:22.322518+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21321,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323360+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:22.323360+0000","last_peered":"2024-09-05T13:24:22.323360+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:22.323360+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29682,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:22.321539+0000","last_peered":"2024-09-05T13:24:22.321539+0000","last_clean":"2024-09-05T13:24:22.321539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:22.321539+0000","last_undegraded":"2024-09-05T13:24:22.321539+0000","last_fullsized":"2024-09-05T13:24:22.321539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21263,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323175+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:22.323175+0000","last_peered":"2024-09-05T13:24:22.323175+0000","last_clean":"2024-09-05T13:24:22.323175+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:22.323175+0000","last_undegraded":"2024-09-05T13:24:22.323175+0000","last_fullsized":"2024-09-05T13:24:22.323175+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321262+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:22.321262+0000","last_peered":"2024-09-05T13:24:22.321262+0000","last_clean":"2024-09-05T13:24:22.321262+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:22.321262+0000","last_undegraded":"2024-09-05T13:24:22.321262+0000","last_fullsized":"2024-09-05T13:24:22.321262+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21254,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321578+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:22.321578+0000","last_peered":"2024-09-05T13:24:22.321578+0000","last_clean":"2024-09-05T13:24:22.321578+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:22.321578+0000","last_undegraded":"2024-09-05T13:24:22.321578+0000","last_fullsized":"2024-09-05T13:24:22.321578+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323557+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:22.323557+0000","last_peered":"2024-09-05T13:24:22.323557+0000","last_clean":"2024-09-05T13:24:22.323557+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:22.323557+0000","last_undegraded":"2024-09-05T13:24:22.323557+0000","last_fullsized":"2024-09-05T13:24:22.323557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":29105,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:46.209158+0000","last_change":"2024-09-05T13:24:20.536499+0000","last_active":"2024-09-05T13:24:46.209158+0000","last_peered":"2024-09-05T13:24:46.209158+0000","last_clean":"2024-09-05T13:13:15.889399+0000","last_became_active":"2024-09-05T13:13:18.533106+0000","last_became_peered":"2024-09-05T13:13:18.533106+0000","last_unstale":"2024-09-05T13:24:46.209158+0000","last_undegraded":"2024-09-05T13:13:17.995842+0000","last_fullsized":"2024-09-05T13:13:17.904290+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":442,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4015,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-442}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323309+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:22.323309+0000","last_peered":"2024-09-05T13:24:22.323309+0000","last_clean":"2024-09-05T13:24:22.323309+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:22.323309+0000","last_undegraded":"2024-09-05T13:24:22.323309+0000","last_fullsized":"2024-09-05T13:24:22.323309+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29839,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323666+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:22.323666+0000","last_peered":"2024-09-05T13:24:22.323666+0000","last_clean":"2024-09-05T13:24:22.323666+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:22.323666+0000","last_undegraded":"2024-09-05T13:24:22.323666+0000","last_fullsized":"2024-09-05T13:24:22.323666+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321540+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:22.321540+0000","last_peered":"2024-09-05T13:24:22.321540+0000","last_clean":"2024-09-05T13:24:22.321540+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:22.321540+0000","last_undegraded":"2024-09-05T13:24:22.321540+0000","last_fullsized":"2024-09-05T13:24:22.321540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29519,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.397315+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:22.397315+0000","last_peered":"2024-09-05T13:24:22.397315+0000","last_clean":"2024-09-05T13:24:22.397315+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:22.397315+0000","last_undegraded":"2024-09-05T13:24:22.397315+0000","last_fullsized":"2024-09-05T13:24:22.397315+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21046,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322680+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:22.322680+0000","last_peered":"2024-09-05T13:24:22.322680+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:22.322680+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321733+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:22.321733+0000","last_peered":"2024-09-05T13:24:22.321733+0000","last_clean":"2024-09-05T13:24:22.321733+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:22.321733+0000","last_undegraded":"2024-09-05T13:24:22.321733+0000","last_fullsized":"2024-09-05T13:24:22.321733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.322660+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:22.322660+0000","last_peered":"2024-09-05T13:24:22.322660+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:22.322660+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.323521+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:22.323521+0000","last_peered":"2024-09-05T13:24:22.323521+0000","last_clean":"2024-09-05T13:24:22.323521+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:22.323521+0000","last_undegraded":"2024-09-05T13:24:22.323521+0000","last_fullsized":"2024-09-05T13:24:22.323521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21251,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321817+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:22.321817+0000","last_peered":"2024-09-05T13:24:22.321817+0000","last_clean":"2024-09-05T13:24:22.321817+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:22.321817+0000","last_undegraded":"2024-09-05T13:24:22.321817+0000","last_fullsized":"2024-09-05T13:24:22.321817+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321878+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:22.321878+0000","last_peered":"2024-09-05T13:24:22.321878+0000","last_clean":"2024-09-05T13:24:22.321878+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:22.321878+0000","last_undegraded":"2024-09-05T13:24:22.321878+0000","last_fullsized":"2024-09-05T13:24:22.321878+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-05T13:24:22.321966+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:22.321966+0000","last_peered":"2024-09-05T13:24:22.321966+0000","last_clean":"2024-09-05T13:24:22.321966+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:22.321966+0000","last_undegraded":"2024-09-05T13:24:22.321966+0000","last_fullsized":"2024-09-05T13:24:22.321966+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:22.323959+0000","last_change":"2024-09-05T13:13:18.552474+0000","last_active":"2024-09-05T13:24:22.323959+0000","last_peered":"2024-09-05T13:24:22.323959+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:22.323959+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4564}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35168,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164085760,"data_stored":164042404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432476,"kb_used_data":89800,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939364,"statfs":{"total":96636764160,"available":96193908736,"internally_reserved":0,"allocated":91955200,"data_stored":91636295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562488,"kb_used_data":71332,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809352,"statfs":{"total":96636764160,"available":96060776448,"internally_reserved":0,"allocated":73043968,"data_stored":72720986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497292,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874548,"statfs":{"total":96636764160,"available":96127537152,"internally_reserved":0,"allocated":520192,"data_stored":164693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:51.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:52.141 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:52.142 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:52.157 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-05T13:24:51.771786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36177,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500608,"kb_used_data":161668,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614912,"statfs":{"total":289910292480,"available":288373669888,"internally_reserved":0,"allocated":165548032,"data_stored":164534633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2098,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001273"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20888,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381487+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:48.381487+0000","last_peered":"2024-09-05T13:24:48.381487+0000","last_clean":"2024-09-05T13:24:48.381487+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:48.381487+0000","last_undegraded":"2024-09-05T13:24:48.381487+0000","last_fullsized":"2024-09-05T13:24:48.381487+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381356+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:48.381356+0000","last_peered":"2024-09-05T13:24:48.381356+0000","last_clean":"2024-09-05T13:24:48.381356+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:48.381356+0000","last_undegraded":"2024-09-05T13:24:48.381356+0000","last_fullsized":"2024-09-05T13:24:48.381356+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381014+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:48.381014+0000","last_peered":"2024-09-05T13:24:48.381014+0000","last_clean":"2024-09-05T13:24:48.381014+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:48.381014+0000","last_undegraded":"2024-09-05T13:24:48.381014+0000","last_fullsized":"2024-09-05T13:24:48.381014+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29842,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381100+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:48.381100+0000","last_peered":"2024-09-05T13:24:48.381100+0000","last_clean":"2024-09-05T13:24:48.381100+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:48.381100+0000","last_undegraded":"2024-09-05T13:24:48.381100+0000","last_fullsized":"2024-09-05T13:24:48.381100+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29521,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.380855+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:48.380855+0000","last_peered":"2024-09-05T13:24:48.380855+0000","last_clean":"2024-09-05T13:24:48.380855+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:48.380855+0000","last_undegraded":"2024-09-05T13:24:48.380855+0000","last_fullsized":"2024-09-05T13:24:48.380855+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381436+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:48.381436+0000","last_peered":"2024-09-05T13:24:48.381436+0000","last_clean":"2024-09-05T13:24:48.381436+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:48.381436+0000","last_undegraded":"2024-09-05T13:24:48.381436+0000","last_fullsized":"2024-09-05T13:24:48.381436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":22497,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:51.216793+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:24:51.216793+0000","last_peered":"2024-09-05T13:24:51.216793+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:51.216793+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":566,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36176,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164073472,"data_stored":164040955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432484,"kb_used_data":89808,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939356,"statfs":{"total":96636764160,"available":96193900544,"internally_reserved":0,"allocated":91963392,"data_stored":91639125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505620,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866220,"statfs":{"total":96636764160,"available":96119009280,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:52.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:52.516 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:52.517 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:52.530 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-05T13:24:51.771786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36177,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500608,"kb_used_data":161668,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614912,"statfs":{"total":289910292480,"available":288373669888,"internally_reserved":0,"allocated":165548032,"data_stored":164534633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2098,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001273"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20888,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381487+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:48.381487+0000","last_peered":"2024-09-05T13:24:48.381487+0000","last_clean":"2024-09-05T13:24:48.381487+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:48.381487+0000","last_undegraded":"2024-09-05T13:24:48.381487+0000","last_fullsized":"2024-09-05T13:24:48.381487+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381356+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:48.381356+0000","last_peered":"2024-09-05T13:24:48.381356+0000","last_clean":"2024-09-05T13:24:48.381356+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:48.381356+0000","last_undegraded":"2024-09-05T13:24:48.381356+0000","last_fullsized":"2024-09-05T13:24:48.381356+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381014+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:48.381014+0000","last_peered":"2024-09-05T13:24:48.381014+0000","last_clean":"2024-09-05T13:24:48.381014+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:48.381014+0000","last_undegraded":"2024-09-05T13:24:48.381014+0000","last_fullsized":"2024-09-05T13:24:48.381014+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29842,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381100+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:48.381100+0000","last_peered":"2024-09-05T13:24:48.381100+0000","last_clean":"2024-09-05T13:24:48.381100+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:48.381100+0000","last_undegraded":"2024-09-05T13:24:48.381100+0000","last_fullsized":"2024-09-05T13:24:48.381100+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29521,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.380855+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:48.380855+0000","last_peered":"2024-09-05T13:24:48.380855+0000","last_clean":"2024-09-05T13:24:48.380855+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:48.380855+0000","last_undegraded":"2024-09-05T13:24:48.380855+0000","last_fullsized":"2024-09-05T13:24:48.380855+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381436+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:48.381436+0000","last_peered":"2024-09-05T13:24:48.381436+0000","last_clean":"2024-09-05T13:24:48.381436+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:48.381436+0000","last_undegraded":"2024-09-05T13:24:48.381436+0000","last_fullsized":"2024-09-05T13:24:48.381436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":22497,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:51.216793+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:24:51.216793+0000","last_peered":"2024-09-05T13:24:51.216793+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:51.216793+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":566,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36176,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164073472,"data_stored":164040955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432484,"kb_used_data":89808,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939356,"statfs":{"total":96636764160,"available":96193900544,"internally_reserved":0,"allocated":91963392,"data_stored":91639125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505620,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866220,"statfs":{"total":96636764160,"available":96119009280,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:52.531 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:52.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21325, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.381887+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:49.381887+0000', 'last_peered': '2024-09-05T13:24:49.381887+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:49.381887+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:52.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:52.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21050, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.382549+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:49.382549+0000', 'last_peered': '2024-09-05T13:24:49.382549+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:49.382549+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:52.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:52.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21323, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.384047+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:49.384047+0000', 'last_peered': '2024-09-05T13:24:49.384047+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:49.384047+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:52.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:52.532 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 22497, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:51.216793+0000', 'last_change': '2024-09-05T13:24:47.705080+0000', 'last_active': '2024-09-05T13:24:51.216793+0000', 'last_peered': '2024-09-05T13:24:51.216793+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:51.216793+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 566, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4001}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:52.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:52.965 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:52.966 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17322/6 objects degraded (288700.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":819,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536622592,"bytes_avail":288373669888,"bytes_total":289910292480,"degraded_objects":17322,"degraded_total":6,"degraded_ratio":2887,"recovering_objects_per_sec":262,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2098,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:52.979 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:52.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:53.338 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:53.339 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:53.352 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2024-09-05T13:24:51.771786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36177,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500608,"kb_used_data":161668,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281614912,"statfs":{"total":289910292480,"available":288373669888,"internally_reserved":0,"allocated":165548032,"data_stored":164534633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1370999814},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2098,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001273"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20888,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381487+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:48.381487+0000","last_peered":"2024-09-05T13:24:48.381487+0000","last_clean":"2024-09-05T13:24:48.381487+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:48.381487+0000","last_undegraded":"2024-09-05T13:24:48.381487+0000","last_fullsized":"2024-09-05T13:24:48.381487+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21125,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381356+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:48.381356+0000","last_peered":"2024-09-05T13:24:48.381356+0000","last_clean":"2024-09-05T13:24:48.381356+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:48.381356+0000","last_undegraded":"2024-09-05T13:24:48.381356+0000","last_fullsized":"2024-09-05T13:24:48.381356+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381014+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:48.381014+0000","last_peered":"2024-09-05T13:24:48.381014+0000","last_clean":"2024-09-05T13:24:48.381014+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:48.381014+0000","last_undegraded":"2024-09-05T13:24:48.381014+0000","last_fullsized":"2024-09-05T13:24:48.381014+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29842,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381100+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:48.381100+0000","last_peered":"2024-09-05T13:24:48.381100+0000","last_clean":"2024-09-05T13:24:48.381100+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:48.381100+0000","last_undegraded":"2024-09-05T13:24:48.381100+0000","last_fullsized":"2024-09-05T13:24:48.381100+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29521,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.380855+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:48.380855+0000","last_peered":"2024-09-05T13:24:48.380855+0000","last_clean":"2024-09-05T13:24:48.380855+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:48.380855+0000","last_undegraded":"2024-09-05T13:24:48.380855+0000","last_fullsized":"2024-09-05T13:24:48.380855+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-05T13:24:48.381436+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:48.381436+0000","last_peered":"2024-09-05T13:24:48.381436+0000","last_clean":"2024-09-05T13:24:48.381436+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:48.381436+0000","last_undegraded":"2024-09-05T13:24:48.381436+0000","last_fullsized":"2024-09-05T13:24:48.381436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":22497,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:51.216793+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:24:51.216793+0000","last_peered":"2024-09-05T13:24:51.216793+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:51.216793+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":566,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36176,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164073472,"data_stored":164040955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432484,"kb_used_data":89808,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93939356,"statfs":{"total":96636764160,"available":96193900544,"internally_reserved":0,"allocated":91963392,"data_stored":91639125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":350875309},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505620,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866220,"statfs":{"total":96636764160,"available":96119009280,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91480064,"data_stored":91476451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:56.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:56.671 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:56.671 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:56.684 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-09-05T13:24:55.772502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36177,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514732,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281600788,"statfs":{"total":289910292480,"available":288359206912,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1403702278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1008,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001406"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":22497,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:51.216793+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:24:51.216793+0000","last_peered":"2024-09-05T13:24:51.216793+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:51.216793+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":566,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36176,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:56.686 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:57.082 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:57.083 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:57.095 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-09-05T13:24:55.772502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36177,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514732,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281600788,"statfs":{"total":289910292480,"available":288359206912,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1403702278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1008,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001406"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":22497,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:51.216793+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:24:51.216793+0000","last_peered":"2024-09-05T13:24:51.216793+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:51.216793+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":566,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36176,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:24:57.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:24:57.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21325, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.381887+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:49.381887+0000', 'last_peered': '2024-09-05T13:24:49.381887+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:49.381887+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:57.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:24:57.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21050, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.382549+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:49.382549+0000', 'last_peered': '2024-09-05T13:24:49.382549+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:49.382549+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:57.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:24:57.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21323, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.384047+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:49.384047+0000', 'last_peered': '2024-09-05T13:24:49.384047+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:49.384047+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:57.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:24:57.098 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 22497, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:51.216793+0000', 'last_change': '2024-09-05T13:24:47.705080+0000', 'last_active': '2024-09-05T13:24:51.216793+0000', 'last_peered': '2024-09-05T13:24:51.216793+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:51.216793+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 566, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4001}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:24:57.098 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:24:57.514 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:57.515 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17322/6 objects degraded (288700.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":824,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1551085568,"bytes_avail":288359206912,"bytes_total":289910292480,"degraded_objects":17322,"degraded_total":6,"degraded_ratio":2887,"recovering_objects_per_sec":125,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1008,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:24:57.529 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:24:57.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:24:57.862 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:24:57.863 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:24:57.875 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-05T13:24:57.772802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37152,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514732,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281600788,"statfs":{"total":289910292480,"available":288359206912,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1403702278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1983,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.392308"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":24447,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:56.214513+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:24:56.214513+0000","last_peered":"2024-09-05T13:24:56.214513+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:56.214513+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3026}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37151,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:00.877 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:01.207 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:01.207 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:01.219 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-09-05T13:24:59.773144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37152,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514732,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281600788,"statfs":{"total":289910292480,"available":288359206912,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1403702278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1983,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.391645"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":24447,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:56.214513+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:24:56.214513+0000","last_peered":"2024-09-05T13:24:56.214513+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:56.214513+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3026}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37151,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:01.221 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:01.555 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:01.555 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:01.569 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-09-05T13:24:59.773144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37152,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514732,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281600788,"statfs":{"total":289910292480,"available":288359206912,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1403702278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1983,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.391645"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":24447,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:56.214513+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:24:56.214513+0000","last_peered":"2024-09-05T13:24:56.214513+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:24:56.214513+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3026}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37151,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497296,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874544,"statfs":{"total":96636764160,"available":96127533056,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:01.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:01.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21325, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.381887+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:49.381887+0000', 'last_peered': '2024-09-05T13:24:49.381887+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:49.381887+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:01.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:01.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21050, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.382549+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:49.382549+0000', 'last_peered': '2024-09-05T13:24:49.382549+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:49.382549+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:01.571 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:25:01.571 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21323, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.384047+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:49.384047+0000', 'last_peered': '2024-09-05T13:24:49.384047+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:49.384047+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:01.571 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:25:01.571 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 24447, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:56.214513+0000', 'last_change': '2024-09-05T13:24:47.705080+0000', 'last_active': '2024-09-05T13:24:56.214513+0000', 'last_peered': '2024-09-05T13:24:56.214513+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:24:56.214513+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3027, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1541, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3026}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:01.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:01.972 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:01.972 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16347/6 objects degraded (272450.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":828,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1551085568,"bytes_avail":288359206912,"bytes_total":289910292480,"degraded_objects":16347,"degraded_total":6,"degraded_ratio":2724.5,"recovering_objects_per_sec":190,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1983,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:01.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:01.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:02.313 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:02.314 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:02.326 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-05T13:25:01.773483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38324,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523052,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1379123,"kb_avail":281592468,"statfs":{"total":289910292480,"available":288350687232,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1412221958},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3155,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002047"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":26791,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:01.215085+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:25:01.215085+0000","last_peered":"2024-09-05T13:25:01.215085+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:25:01.215085+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2713,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38323,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:05.328 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:05.663 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:05.663 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:05.677 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-05T13:25:03.773866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38324,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523052,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1379123,"kb_avail":281592468,"statfs":{"total":289910292480,"available":288350687232,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1412221958},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002080"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28605,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384736+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384736+0000","last_peered":"2024-09-05T13:24:49.384736+0000","last_clean":"2024-09-05T13:24:49.384736+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384736+0000","last_undegraded":"2024-09-05T13:24:49.384736+0000","last_fullsized":"2024-09-05T13:24:49.384736+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":26791,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:01.215085+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:25:01.215085+0000","last_peered":"2024-09-05T13:25:01.215085+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:25:01.215085+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2713,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38323,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505616,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866224,"statfs":{"total":96636764160,"available":96119013376,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:05.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:06.014 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:06.028 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-05T13:25:05.774130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38324,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518892,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281596628,"statfs":{"total":289910292480,"available":288354947072,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1407962118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001937"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28606,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384777+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384777+0000","last_peered":"2024-09-05T13:24:49.384777+0000","last_clean":"2024-09-05T13:24:49.384777+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384777+0000","last_undegraded":"2024-09-05T13:24:49.384777+0000","last_fullsized":"2024-09-05T13:24:49.384777+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":26791,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:01.215085+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:25:01.215085+0000","last_peered":"2024-09-05T13:25:01.215085+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:25:01.215085+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2713,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38323,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501456,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870384,"statfs":{"total":96636764160,"available":96123273216,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:06.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:06.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21325, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.381887+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:49.381887+0000', 'last_peered': '2024-09-05T13:24:49.381887+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:49.381887+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:06.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:06.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21050, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.382549+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:49.382549+0000', 'last_peered': '2024-09-05T13:24:49.382549+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:49.382549+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:06.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:25:06.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21323, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.384047+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:49.384047+0000', 'last_peered': '2024-09-05T13:24:49.384047+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:49.384047+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:06.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:25:06.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 26791, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:01.215085+0000', 'last_change': '2024-09-05T13:24:47.705080+0000', 'last_active': '2024-09-05T13:25:01.215085+0000', 'last_peered': '2024-09-05T13:25:01.215085+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:25:01.215085+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1855, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2713, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1854}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:06.031 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:06.419 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:06.419 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15175/6 objects degraded (252916.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":833,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1559605248,"bytes_avail":288350687232,"bytes_total":289910292480,"degraded_objects":15175,"degraded_total":6,"degraded_ratio":2529.1666666666665,"recovering_objects_per_sec":178,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2147,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:06.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:06.434 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:06.730 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:06.731 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:06.743 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-05T13:25:05.774130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38324,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518892,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281596628,"statfs":{"total":289910292480,"available":288354947072,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1407962118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2147,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001937"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28606,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384777+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384777+0000","last_peered":"2024-09-05T13:24:49.384777+0000","last_clean":"2024-09-05T13:24:49.384777+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384777+0000","last_undegraded":"2024-09-05T13:24:49.384777+0000","last_fullsized":"2024-09-05T13:24:49.384777+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21267,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381949+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381949+0000","last_peered":"2024-09-05T13:24:49.381949+0000","last_clean":"2024-09-05T13:24:49.381949+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381949+0000","last_undegraded":"2024-09-05T13:24:49.381949+0000","last_fullsized":"2024-09-05T13:24:49.381949+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":26791,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:01.215085+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:25:01.215085+0000","last_peered":"2024-09-05T13:25:01.215085+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:25:01.215085+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2713,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38323,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501456,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870384,"statfs":{"total":96636764160,"available":96123273216,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:09.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:10.097 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:10.111 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-05T13:25:09.774732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39474,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518892,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281596628,"statfs":{"total":289910292480,"available":288354947072,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1407962118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001587"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28606,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384777+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384777+0000","last_peered":"2024-09-05T13:24:49.384777+0000","last_clean":"2024-09-05T13:24:49.384777+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384777+0000","last_undegraded":"2024-09-05T13:24:49.384777+0000","last_fullsized":"2024-09-05T13:24:49.384777+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21268,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381986+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381986+0000","last_peered":"2024-09-05T13:24:49.381986+0000","last_clean":"2024-09-05T13:24:49.381986+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381986+0000","last_undegraded":"2024-09-05T13:24:49.381986+0000","last_fullsized":"2024-09-05T13:24:49.381986+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":29091,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:06.214474+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:25:06.214474+0000","last_peered":"2024-09-05T13:25:06.214474+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:25:06.214474+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3863,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39473,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501456,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870384,"statfs":{"total":96636764160,"available":96123273216,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:10.112 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:10.431 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:10.432 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:10.445 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-05T13:25:09.774732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39474,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518892,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281596628,"statfs":{"total":289910292480,"available":288354947072,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1407962118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2322,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001587"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28606,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384777+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384777+0000","last_peered":"2024-09-05T13:24:49.384777+0000","last_clean":"2024-09-05T13:24:49.384777+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384777+0000","last_undegraded":"2024-09-05T13:24:49.384777+0000","last_fullsized":"2024-09-05T13:24:49.384777+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21268,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381986+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381986+0000","last_peered":"2024-09-05T13:24:49.381986+0000","last_clean":"2024-09-05T13:24:49.381986+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381986+0000","last_undegraded":"2024-09-05T13:24:49.381986+0000","last_fullsized":"2024-09-05T13:24:49.381986+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":29091,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:06.214474+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:25:06.214474+0000","last_peered":"2024-09-05T13:25:06.214474+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:25:06.214474+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3863,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39473,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501456,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870384,"statfs":{"total":96636764160,"available":96123273216,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:10.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:10.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21325, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.381887+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:24:49.381887+0000', 'last_peered': '2024-09-05T13:24:49.381887+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:24:49.381887+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:10.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:10.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21050, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.382549+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:24:49.382549+0000', 'last_peered': '2024-09-05T13:24:49.382549+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:24:49.382549+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:10.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:25:10.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 21323, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:24:49.384047+0000', 'last_change': '2024-09-05T13:13:18.554206+0000', 'last_active': '2024-09-05T13:24:49.384047+0000', 'last_peered': '2024-09-05T13:24:49.384047+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:24:49.384047+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4447, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:10.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-05T13:25:10.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'21192", 'reported_seq': 29091, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:06.214474+0000', 'last_change': '2024-09-05T13:24:47.705080+0000', 'last_active': '2024-09-05T13:25:06.214474+0000', 'last_peered': '2024-09-05T13:25:06.214474+0000', 'last_clean': '2024-09-05T13:13:15.904752+0000', 'last_became_active': '2024-09-05T13:13:18.533436+0000', 'last_became_peered': '2024-09-05T13:13:18.533436+0000', 'last_unstale': '2024-09-05T13:25:06.214474+0000', 'last_undegraded': '2024-09-05T13:13:18.007902+0000', 'last_fullsized': '2024-09-05T13:13:17.903386+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3398", 'last_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:12:59.497416+0000', 'objects_scrubbed': 3321, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1460, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 705, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17860, 'num_write_kb': 29080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3863, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -704}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:10.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:10.951 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:10.951 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14025/6 objects degraded (233750.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":837,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1555345408,"bytes_avail":288354947072,"bytes_total":289910292480,"degraded_objects":14025,"degraded_total":6,"degraded_ratio":2337.5,"recovering_objects_per_sec":232,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2322,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:10.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:10.965 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:11.289 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:11.289 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:11.301 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-05T13:25:10.443043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39474,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518892,"kb_used_data":143860,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281596628,"statfs":{"total":289910292480,"available":288354947072,"internally_reserved":0,"allocated":147312640,"data_stored":146293858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1407962118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.669560"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29928,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384006+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:24:49.384006+0000","last_peered":"2024-09-05T13:24:49.384006+0000","last_clean":"2024-09-05T13:24:49.384006+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:24:49.384006+0000","last_undegraded":"2024-09-05T13:24:49.384006+0000","last_fullsized":"2024-09-05T13:24:49.384006+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28606,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384777+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:24:49.384777+0000","last_peered":"2024-09-05T13:24:49.384777+0000","last_clean":"2024-09-05T13:24:49.384777+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:24:49.384777+0000","last_undegraded":"2024-09-05T13:24:49.384777+0000","last_fullsized":"2024-09-05T13:24:49.384777+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21049,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382287+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:24:49.382287+0000","last_peered":"2024-09-05T13:24:49.382287+0000","last_clean":"2024-09-05T13:24:49.382287+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:24:49.382287+0000","last_undegraded":"2024-09-05T13:24:49.382287+0000","last_fullsized":"2024-09-05T13:24:49.382287+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21325,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.381887+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:24:49.381887+0000","last_peered":"2024-09-05T13:24:49.381887+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:24:49.381887+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29686,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384539+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:24:49.384539+0000","last_peered":"2024-09-05T13:24:49.384539+0000","last_clean":"2024-09-05T13:24:49.384539+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:24:49.384539+0000","last_undegraded":"2024-09-05T13:24:49.384539+0000","last_fullsized":"2024-09-05T13:24:49.384539+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21268,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381986+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:24:49.381986+0000","last_peered":"2024-09-05T13:24:49.381986+0000","last_clean":"2024-09-05T13:24:49.381986+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:24:49.381986+0000","last_undegraded":"2024-09-05T13:24:49.381986+0000","last_fullsized":"2024-09-05T13:24:49.381986+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384356+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:24:49.384356+0000","last_peered":"2024-09-05T13:24:49.384356+0000","last_clean":"2024-09-05T13:24:49.384356+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:24:49.384356+0000","last_undegraded":"2024-09-05T13:24:49.384356+0000","last_fullsized":"2024-09-05T13:24:49.384356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21258,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384511+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:24:49.384511+0000","last_peered":"2024-09-05T13:24:49.384511+0000","last_clean":"2024-09-05T13:24:49.384511+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:24:49.384511+0000","last_undegraded":"2024-09-05T13:24:49.384511+0000","last_fullsized":"2024-09-05T13:24:49.384511+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30000,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.438148+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:24:49.438148+0000","last_peered":"2024-09-05T13:24:49.438148+0000","last_clean":"2024-09-05T13:24:49.438148+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:24:49.438148+0000","last_undegraded":"2024-09-05T13:24:49.438148+0000","last_fullsized":"2024-09-05T13:24:49.438148+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":136,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381773+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:24:49.381773+0000","last_peered":"2024-09-05T13:24:49.381773+0000","last_clean":"2024-09-05T13:24:49.381773+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:24:49.381773+0000","last_undegraded":"2024-09-05T13:24:49.381773+0000","last_fullsized":"2024-09-05T13:24:49.381773+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384429+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:24:49.384429+0000","last_peered":"2024-09-05T13:24:49.384429+0000","last_clean":"2024-09-05T13:24:49.384429+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:24:49.384429+0000","last_undegraded":"2024-09-05T13:24:49.384429+0000","last_fullsized":"2024-09-05T13:24:49.384429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21050,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.382549+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:24:49.382549+0000","last_peered":"2024-09-05T13:24:49.382549+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:24:49.382549+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384670+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:24:49.384670+0000","last_peered":"2024-09-05T13:24:49.384670+0000","last_clean":"2024-09-05T13:24:49.384670+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:24:49.384670+0000","last_undegraded":"2024-09-05T13:24:49.384670+0000","last_fullsized":"2024-09-05T13:24:49.384670+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21323,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:24:49.384047+0000","last_change":"2024-09-05T13:13:18.554206+0000","last_active":"2024-09-05T13:24:49.384047+0000","last_peered":"2024-09-05T13:24:49.384047+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:24:49.384047+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4447,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4444}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384821+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:24:49.384821+0000","last_peered":"2024-09-05T13:24:49.384821+0000","last_clean":"2024-09-05T13:24:49.384821+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:24:49.384821+0000","last_undegraded":"2024-09-05T13:24:49.384821+0000","last_fullsized":"2024-09-05T13:24:49.384821+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384437+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:24:49.384437+0000","last_peered":"2024-09-05T13:24:49.384437+0000","last_clean":"2024-09-05T13:24:49.384437+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:24:49.384437+0000","last_undegraded":"2024-09-05T13:24:49.384437+0000","last_fullsized":"2024-09-05T13:24:49.384437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.384601+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:24:49.384601+0000","last_peered":"2024-09-05T13:24:49.384601+0000","last_clean":"2024-09-05T13:24:49.384601+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:24:49.384601+0000","last_undegraded":"2024-09-05T13:24:49.384601+0000","last_fullsized":"2024-09-05T13:24:49.384601+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":29091,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:06.214474+0000","last_change":"2024-09-05T13:24:47.705080+0000","last_active":"2024-09-05T13:25:06.214474+0000","last_peered":"2024-09-05T13:25:06.214474+0000","last_clean":"2024-09-05T13:13:15.904752+0000","last_became_active":"2024-09-05T13:13:18.533436+0000","last_became_peered":"2024-09-05T13:13:18.533436+0000","last_unstale":"2024-09-05T13:25:06.214474+0000","last_undegraded":"2024-09-05T13:13:18.007902+0000","last_fullsized":"2024-09-05T13:13:17.903386+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":705,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3863,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-704}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39473,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145829888,"data_stored":145797371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562504,"kb_used_data":71348,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809336,"statfs":{"total":96636764160,"available":96060760064,"internally_reserved":0,"allocated":73060352,"data_stored":72726625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501456,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870384,"statfs":{"total":96636764160,"available":96123273216,"internally_reserved":0,"allocated":524288,"data_stored":168883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:14.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:14.648 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:14.648 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:14.661 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-09-05T13:25:13.775473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40535,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527216,"kb_used_data":143864,"kb_used_omap":12,"kb_used_meta":1383283,"kb_avail":281588304,"statfs":{"total":289910292480,"available":288346423296,"internally_reserved":0,"allocated":147316736,"data_stored":146298019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1416481798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001342"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446541+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446541+0000","last_peered":"2024-09-05T13:25:10.446541+0000","last_clean":"2024-09-05T13:25:10.446541+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446541+0000","last_undegraded":"2024-09-05T13:25:10.446541+0000","last_fullsized":"2024-09-05T13:25:10.446541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":22040,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:11.218333+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:11.218333+0000","last_peered":"2024-09-05T13:25:11.218333+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:11.218333+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40534,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145817600,"data_stored":145795922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562512,"kb_used_data":71356,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809328,"statfs":{"total":96636764160,"available":96060751872,"internally_reserved":0,"allocated":73068544,"data_stored":72729430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":520192,"data_stored":170239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:14.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:15.006 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:15.020 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-09-05T13:25:13.775473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40535,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527216,"kb_used_data":143864,"kb_used_omap":12,"kb_used_meta":1383283,"kb_avail":281588304,"statfs":{"total":289910292480,"available":288346423296,"internally_reserved":0,"allocated":147316736,"data_stored":146298019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1416481798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001342"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20890,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382496+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:24:49.382496+0000","last_peered":"2024-09-05T13:24:49.382496+0000","last_clean":"2024-09-05T13:24:49.382496+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:24:49.382496+0000","last_undegraded":"2024-09-05T13:24:49.382496+0000","last_fullsized":"2024-09-05T13:24:49.382496+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382331+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:24:49.382331+0000","last_peered":"2024-09-05T13:24:49.382331+0000","last_clean":"2024-09-05T13:24:49.382331+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:24:49.382331+0000","last_undegraded":"2024-09-05T13:24:49.382331+0000","last_fullsized":"2024-09-05T13:24:49.382331+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446541+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446541+0000","last_peered":"2024-09-05T13:25:10.446541+0000","last_clean":"2024-09-05T13:25:10.446541+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446541+0000","last_undegraded":"2024-09-05T13:25:10.446541+0000","last_fullsized":"2024-09-05T13:25:10.446541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381643+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:24:49.381643+0000","last_peered":"2024-09-05T13:24:49.381643+0000","last_clean":"2024-09-05T13:24:49.381643+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:24:49.381643+0000","last_undegraded":"2024-09-05T13:24:49.381643+0000","last_fullsized":"2024-09-05T13:24:49.381643+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29844,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381940+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:24:49.381940+0000","last_peered":"2024-09-05T13:24:49.381940+0000","last_clean":"2024-09-05T13:24:49.381940+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:24:49.381940+0000","last_undegraded":"2024-09-05T13:24:49.381940+0000","last_fullsized":"2024-09-05T13:24:49.381940+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29523,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.381792+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:24:49.381792+0000","last_peered":"2024-09-05T13:24:49.381792+0000","last_clean":"2024-09-05T13:24:49.381792+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:24:49.381792+0000","last_undegraded":"2024-09-05T13:24:49.381792+0000","last_fullsized":"2024-09-05T13:24:49.381792+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":22040,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:11.218333+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:11.218333+0000","last_peered":"2024-09-05T13:25:11.218333+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:11.218333+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-05T13:24:49.382140+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:24:49.382140+0000","last_peered":"2024-09-05T13:24:49.382140+0000","last_clean":"2024-09-05T13:24:49.382140+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:24:49.382140+0000","last_undegraded":"2024-09-05T13:24:49.382140+0000","last_fullsized":"2024-09-05T13:24:49.382140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40534,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145817600,"data_stored":145795922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454932,"kb_used_data":72000,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93916908,"statfs":{"total":96636764160,"available":96170913792,"internally_reserved":0,"allocated":73728000,"data_stored":73398350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562512,"kb_used_data":71356,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809328,"statfs":{"total":96636764160,"available":96060751872,"internally_reserved":0,"allocated":73068544,"data_stored":72729430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":520192,"data_stored":170239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73236480,"data_stored":73232867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:15.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:15.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21329, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:10.445879+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:10.445879+0000', 'last_peered': '2024-09-05T13:25:10.445879+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:10.445879+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:15.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:15.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21054, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:10.446394+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:25:10.446394+0000', 'last_peered': '2024-09-05T13:25:10.446394+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:10.446394+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:15.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:25:15.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 22040, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:11.218333+0000', 'last_change': '2024-09-05T13:25:08.703987+0000', 'last_active': '2024-09-05T13:25:11.218333+0000', 'last_peered': '2024-09-05T13:25:11.218333+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:25:11.218333+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 356, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4091}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:15.022 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:15.475 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:15.475 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12964/6 objects degraded (216066.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":842,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1563869184,"bytes_avail":288346423296,"bytes_total":289910292480,"degraded_objects":12964,"degraded_total":6,"degraded_ratio":2160.6666666666665,"recovering_objects_per_sec":276,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2211,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:15.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:15.489 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:15.824 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:15.825 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:15.838 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-05T13:25:15.775760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40535,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1508968,"kb_used_data":125616,"kb_used_omap":12,"kb_used_meta":1383283,"kb_avail":281606552,"statfs":{"total":289910292480,"available":288365109248,"internally_reserved":0,"allocated":128630784,"data_stored":127604885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1416481798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1061,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001333"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446541+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446541+0000","last_peered":"2024-09-05T13:25:10.446541+0000","last_clean":"2024-09-05T13:25:10.446541+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446541+0000","last_undegraded":"2024-09-05T13:25:10.446541+0000","last_fullsized":"2024-09-05T13:25:10.446541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":22040,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:11.218333+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:11.218333+0000","last_peered":"2024-09-05T13:25:11.218333+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:11.218333+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40534,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562512,"kb_used_data":71356,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809328,"statfs":{"total":96636764160,"available":96060751872,"internally_reserved":0,"allocated":73068544,"data_stored":72729430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":520192,"data_stored":170239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:18.841 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:19.176 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:19.177 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:19.189 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-05T13:25:17.776109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41507,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500664,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281614856,"statfs":{"total":289910292480,"available":288373612544,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1407962118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2033,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.331571"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446541+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446541+0000","last_peered":"2024-09-05T13:25:10.446541+0000","last_clean":"2024-09-05T13:25:10.446541+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446541+0000","last_undegraded":"2024-09-05T13:25:10.446541+0000","last_fullsized":"2024-09-05T13:25:10.446541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":23984,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:16.228257+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:16.228257+0000","last_peered":"2024-09-05T13:25:16.228257+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:16.228257+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41506,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501460,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870380,"statfs":{"total":96636764160,"available":96123269120,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:19.191 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:19.611 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:19.612 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:19.626 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-05T13:25:17.776109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41507,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500664,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281614856,"statfs":{"total":289910292480,"available":288373612544,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1407962118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2033,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.331571"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446541+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446541+0000","last_peered":"2024-09-05T13:25:10.446541+0000","last_clean":"2024-09-05T13:25:10.446541+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446541+0000","last_undegraded":"2024-09-05T13:25:10.446541+0000","last_fullsized":"2024-09-05T13:25:10.446541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":23984,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:16.228257+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:16.228257+0000","last_peered":"2024-09-05T13:25:16.228257+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:16.228257+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41506,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501460,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870380,"statfs":{"total":96636764160,"available":96123269120,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:19.627 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:19.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21329, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:10.445879+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:10.445879+0000', 'last_peered': '2024-09-05T13:25:10.445879+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:10.445879+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:19.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:19.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21054, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:10.446394+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:25:10.446394+0000', 'last_peered': '2024-09-05T13:25:10.446394+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:10.446394+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:19.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:25:19.628 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 23984, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:16.228257+0000', 'last_change': '2024-09-05T13:25:08.703987+0000', 'last_active': '2024-09-05T13:25:16.228257+0000', 'last_peered': '2024-09-05T13:25:16.228257+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:25:16.228257+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3119, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1328, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3119}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:19.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:20.084 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:20.084 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11992/6 objects degraded (199866.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":846,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1536679936,"bytes_avail":288373612544,"bytes_total":289910292480,"degraded_objects":11992,"degraded_total":6,"degraded_ratio":1998.666666666667,"recovering_objects_per_sec":244,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2033,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:20.099 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:20.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:20.464 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:20.464 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:20.478 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-05T13:25:19.776435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41507,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500664,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281614856,"statfs":{"total":289910292480,"available":288373612544,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1407962118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2033,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001702"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446541+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446541+0000","last_peered":"2024-09-05T13:25:10.446541+0000","last_clean":"2024-09-05T13:25:10.446541+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446541+0000","last_undegraded":"2024-09-05T13:25:10.446541+0000","last_fullsized":"2024-09-05T13:25:10.446541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":23984,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:16.228257+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:16.228257+0000","last_peered":"2024-09-05T13:25:16.228257+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:16.228257+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41506,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501460,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870380,"statfs":{"total":96636764160,"available":96123269120,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:23.480 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:23.831 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:23.831 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:23.844 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-05T13:25:23.777108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42681,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527032,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1401331,"kb_avail":281588488,"statfs":{"total":289910292480,"available":288346611712,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1434962950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002059"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446586+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446586+0000","last_peered":"2024-09-05T13:25:10.446586+0000","last_clean":"2024-09-05T13:25:10.446586+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446586+0000","last_undegraded":"2024-09-05T13:25:10.446586+0000","last_fullsized":"2024-09-05T13:25:10.446586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":26332,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:21.228225+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:21.228225+0000","last_peered":"2024-09-05T13:25:21.228225+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:21.228225+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2502,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42680,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527828,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844012,"statfs":{"total":96636764160,"available":96096268288,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:23.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:24.196 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:24.197 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:24.210 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-05T13:25:23.777108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42681,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527032,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1401331,"kb_avail":281588488,"statfs":{"total":289910292480,"available":288346611712,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1434962950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002059"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446586+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446586+0000","last_peered":"2024-09-05T13:25:10.446586+0000","last_clean":"2024-09-05T13:25:10.446586+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446586+0000","last_undegraded":"2024-09-05T13:25:10.446586+0000","last_fullsized":"2024-09-05T13:25:10.446586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":26332,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:21.228225+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:21.228225+0000","last_peered":"2024-09-05T13:25:21.228225+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:21.228225+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2502,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42680,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527828,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844012,"statfs":{"total":96636764160,"available":96096268288,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:24.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:24.211 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21329, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:10.445879+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:10.445879+0000', 'last_peered': '2024-09-05T13:25:10.445879+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:10.445879+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:24.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:24.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21054, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:10.446394+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:25:10.446394+0000', 'last_peered': '2024-09-05T13:25:10.446394+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:10.446394+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:24.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:25:24.212 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 26332, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:21.228225+0000', 'last_change': '2024-09-05T13:25:08.703987+0000', 'last_active': '2024-09-05T13:25:21.228225+0000', 'last_peered': '2024-09-05T13:25:21.228225+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:25:21.228225+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1945, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2502, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1945}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:24.212 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:24.637 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:24.638 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10818/6 objects degraded (180300.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":851,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1563680768,"bytes_avail":288346611712,"bytes_total":289910292480,"degraded_objects":10818,"degraded_total":6,"degraded_ratio":1803,"recovering_objects_per_sec":178,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2146,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:24.651 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:24.652 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:24.997 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:25.009 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-05T13:25:23.777108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42681,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527032,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1401331,"kb_avail":281588488,"statfs":{"total":289910292480,"available":288346611712,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1434962950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002059"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446586+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446586+0000","last_peered":"2024-09-05T13:25:10.446586+0000","last_clean":"2024-09-05T13:25:10.446586+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446586+0000","last_undegraded":"2024-09-05T13:25:10.446586+0000","last_fullsized":"2024-09-05T13:25:10.446586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":26332,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:21.228225+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:21.228225+0000","last_peered":"2024-09-05T13:25:21.228225+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:21.228225+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1945,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2502,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42680,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527828,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844012,"statfs":{"total":96636764160,"available":96096268288,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:28.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:28.334 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:28.335 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:28.348 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-05T13:25:27.777805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44248,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522872,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281592648,"statfs":{"total":289910292480,"available":288350871552,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001696"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446586+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446586+0000","last_peered":"2024-09-05T13:25:10.446586+0000","last_clean":"2024-09-05T13:25:10.446586+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446586+0000","last_undegraded":"2024-09-05T13:25:10.446586+0000","last_fullsized":"2024-09-05T13:25:10.446586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":29466,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:26.229257+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:26.229257+0000","last_peered":"2024-09-05T13:25:26.229257+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:26.229257+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4069,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44247,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523668,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848172,"statfs":{"total":96636764160,"available":96100528128,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:28.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:28.743 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:28.758 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-09-05T13:25:28.504829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44248,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522872,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281592648,"statfs":{"total":289910292480,"available":288350871552,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.728394"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446586+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446586+0000","last_peered":"2024-09-05T13:25:10.446586+0000","last_clean":"2024-09-05T13:25:10.446586+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446586+0000","last_undegraded":"2024-09-05T13:25:10.446586+0000","last_fullsized":"2024-09-05T13:25:10.446586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":29466,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:26.229257+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:26.229257+0000","last_peered":"2024-09-05T13:25:26.229257+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:26.229257+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4069,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44247,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523668,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848172,"statfs":{"total":96636764160,"available":96100528128,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:28.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:28.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21329, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:10.445879+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:10.445879+0000', 'last_peered': '2024-09-05T13:25:10.445879+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:10.445879+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:28.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:28.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 21054, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:10.446394+0000', 'last_change': '2024-09-05T13:13:18.560424+0000', 'last_active': '2024-09-05T13:25:10.446394+0000', 'last_peered': '2024-09-05T13:25:10.446394+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:10.446394+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:28.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-05T13:25:28.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'21104", 'reported_seq': 29466, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:26.229257+0000', 'last_change': '2024-09-05T13:25:08.703987+0000', 'last_active': '2024-09-05T13:25:26.229257+0000', 'last_peered': '2024-09-05T13:25:26.229257+0000', 'last_clean': '2024-09-05T13:13:15.904984+0000', 'last_became_active': '2024-09-05T13:13:18.534772+0000', 'last_became_peered': '2024-09-05T13:13:18.534772+0000', 'last_unstale': '2024-09-05T13:25:26.229257+0000', 'last_undegraded': '2024-09-05T13:13:17.975037+0000', 'last_fullsized': '2024-09-05T13:13:17.903748+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'4156", 'last_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:11.658445+0000', 'objects_scrubbed': 4093, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1558, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 378, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17003, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4069, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -378}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:28.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:29.149 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:29.150 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9251/6 objects degraded (154183.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":856,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1559420928,"bytes_avail":288350871552,"bytes_total":289910292480,"degraded_objects":9251,"degraded_total":6,"degraded_ratio":1541.833333333333,"recovering_objects_per_sec":274,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:29.165 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:29.166 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:29.470 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:29.471 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:29.483 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-09-05T13:25:28.504829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44248,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522872,"kb_used_data":125632,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281592648,"statfs":{"total":289910292480,"available":288350871552,"internally_reserved":0,"allocated":128647168,"data_stored":127610453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2741,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2741,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.728394"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29932,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446417+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:10.446417+0000","last_peered":"2024-09-05T13:25:10.446417+0000","last_clean":"2024-09-05T13:25:10.446417+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:10.446417+0000","last_undegraded":"2024-09-05T13:25:10.446417+0000","last_fullsized":"2024-09-05T13:25:10.446417+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20894,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444858+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:10.444858+0000","last_peered":"2024-09-05T13:25:10.444858+0000","last_clean":"2024-09-05T13:25:10.444858+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:10.444858+0000","last_undegraded":"2024-09-05T13:25:10.444858+0000","last_fullsized":"2024-09-05T13:25:10.444858+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28610,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446517+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:10.446517+0000","last_peered":"2024-09-05T13:25:10.446517+0000","last_clean":"2024-09-05T13:25:10.446517+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:10.446517+0000","last_undegraded":"2024-09-05T13:25:10.446517+0000","last_fullsized":"2024-09-05T13:25:10.446517+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444617+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:10.444617+0000","last_peered":"2024-09-05T13:25:10.444617+0000","last_clean":"2024-09-05T13:25:10.444617+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:10.444617+0000","last_undegraded":"2024-09-05T13:25:10.444617+0000","last_fullsized":"2024-09-05T13:25:10.444617+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21053,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446158+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:10.446158+0000","last_peered":"2024-09-05T13:25:10.446158+0000","last_clean":"2024-09-05T13:25:10.446158+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:10.446158+0000","last_undegraded":"2024-09-05T13:25:10.446158+0000","last_fullsized":"2024-09-05T13:25:10.446158+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21329,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.445879+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:10.445879+0000","last_peered":"2024-09-05T13:25:10.445879+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:10.445879+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29690,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446770+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:10.446770+0000","last_peered":"2024-09-05T13:25:10.446770+0000","last_clean":"2024-09-05T13:25:10.446770+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:10.446770+0000","last_undegraded":"2024-09-05T13:25:10.446770+0000","last_fullsized":"2024-09-05T13:25:10.446770+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21272,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445873+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:10.445873+0000","last_peered":"2024-09-05T13:25:10.445873+0000","last_clean":"2024-09-05T13:25:10.445873+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:10.445873+0000","last_undegraded":"2024-09-05T13:25:10.445873+0000","last_fullsized":"2024-09-05T13:25:10.445873+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446586+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:10.446586+0000","last_peered":"2024-09-05T13:25:10.446586+0000","last_clean":"2024-09-05T13:25:10.446586+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:10.446586+0000","last_undegraded":"2024-09-05T13:25:10.446586+0000","last_fullsized":"2024-09-05T13:25:10.446586+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21262,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446623+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:10.446623+0000","last_peered":"2024-09-05T13:25:10.446623+0000","last_clean":"2024-09-05T13:25:10.446623+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:10.446623+0000","last_undegraded":"2024-09-05T13:25:10.446623+0000","last_fullsized":"2024-09-05T13:25:10.446623+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444250+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:10.444250+0000","last_peered":"2024-09-05T13:25:10.444250+0000","last_clean":"2024-09-05T13:25:10.444250+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:10.444250+0000","last_undegraded":"2024-09-05T13:25:10.444250+0000","last_fullsized":"2024-09-05T13:25:10.444250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30004,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445982+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:10.445982+0000","last_peered":"2024-09-05T13:25:10.445982+0000","last_clean":"2024-09-05T13:25:10.445982+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:10.445982+0000","last_undegraded":"2024-09-05T13:25:10.445982+0000","last_fullsized":"2024-09-05T13:25:10.445982+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":140,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.445791+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:10.445791+0000","last_peered":"2024-09-05T13:25:10.445791+0000","last_clean":"2024-09-05T13:25:10.445791+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:10.445791+0000","last_undegraded":"2024-09-05T13:25:10.445791+0000","last_fullsized":"2024-09-05T13:25:10.445791+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29848,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444343+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:10.444343+0000","last_peered":"2024-09-05T13:25:10.444343+0000","last_clean":"2024-09-05T13:25:10.444343+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:10.444343+0000","last_undegraded":"2024-09-05T13:25:10.444343+0000","last_fullsized":"2024-09-05T13:25:10.444343+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446489+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:10.446489+0000","last_peered":"2024-09-05T13:25:10.446489+0000","last_clean":"2024-09-05T13:25:10.446489+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:10.446489+0000","last_undegraded":"2024-09-05T13:25:10.446489+0000","last_fullsized":"2024-09-05T13:25:10.446489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29527,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444959+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:10.444959+0000","last_peered":"2024-09-05T13:25:10.444959+0000","last_clean":"2024-09-05T13:25:10.444959+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:10.444959+0000","last_undegraded":"2024-09-05T13:25:10.444959+0000","last_fullsized":"2024-09-05T13:25:10.444959+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21054,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:10.446394+0000","last_change":"2024-09-05T13:13:18.560424+0000","last_active":"2024-09-05T13:25:10.446394+0000","last_peered":"2024-09-05T13:25:10.446394+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:10.446394+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446835+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:10.446835+0000","last_peered":"2024-09-05T13:25:10.446835+0000","last_clean":"2024-09-05T13:25:10.446835+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:10.446835+0000","last_undegraded":"2024-09-05T13:25:10.446835+0000","last_fullsized":"2024-09-05T13:25:10.446835+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":29466,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:26.229257+0000","last_change":"2024-09-05T13:25:08.703987+0000","last_active":"2024-09-05T13:25:26.229257+0000","last_peered":"2024-09-05T13:25:26.229257+0000","last_clean":"2024-09-05T13:13:15.904984+0000","last_became_active":"2024-09-05T13:13:18.534772+0000","last_became_peered":"2024-09-05T13:13:18.534772+0000","last_unstale":"2024-09-05T13:25:26.229257+0000","last_undegraded":"2024-09-05T13:13:17.975037+0000","last_fullsized":"2024-09-05T13:13:17.903748+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4069,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.444690+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:10.444690+0000","last_peered":"2024-09-05T13:25:10.444690+0000","last_clean":"2024-09-05T13:25:10.444690+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:10.444690+0000","last_undegraded":"2024-09-05T13:25:10.444690+0000","last_fullsized":"2024-09-05T13:25:10.444690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21259,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446626+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:10.446626+0000","last_peered":"2024-09-05T13:25:10.446626+0000","last_clean":"2024-09-05T13:25:10.446626+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:10.446626+0000","last_undegraded":"2024-09-05T13:25:10.446626+0000","last_fullsized":"2024-09-05T13:25:10.446626+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446607+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:10.446607+0000","last_peered":"2024-09-05T13:25:10.446607+0000","last_clean":"2024-09-05T13:25:10.446607+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:10.446607+0000","last_undegraded":"2024-09-05T13:25:10.446607+0000","last_fullsized":"2024-09-05T13:25:10.446607+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.446862+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:10.446862+0000","last_peered":"2024-09-05T13:25:10.446862+0000","last_clean":"2024-09-05T13:25:10.446862+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:10.446862+0000","last_undegraded":"2024-09-05T13:25:10.446862+0000","last_fullsized":"2024-09-05T13:25:10.446862+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21352,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-05T13:25:10.552349+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:10.552349+0000","last_peered":"2024-09-05T13:25:10.552349+0000","last_clean":"2024-09-05T13:25:10.552349+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:10.552349+0000","last_undegraded":"2024-09-05T13:25:10.552349+0000","last_fullsized":"2024-09-05T13:25:10.552349+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44247,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":127115264,"data_stored":127097199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436684,"kb_used_data":53752,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93935156,"statfs":{"total":96636764160,"available":96189599744,"internally_reserved":0,"allocated":55042048,"data_stored":54705216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562520,"kb_used_data":71364,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809320,"statfs":{"total":96636764160,"available":96060743680,"internally_reserved":0,"allocated":73076736,"data_stored":72732214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523668,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848172,"statfs":{"total":96636764160,"available":96100528128,"internally_reserved":0,"allocated":528384,"data_stored":173023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54534144,"data_stored":54534144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:32.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:32.807 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:32.807 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:32.819 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-05T13:25:31.778505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45158,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507884,"kb_used_data":118840,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281607636,"statfs":{"total":289910292480,"available":288366219264,"internally_reserved":0,"allocated":121692160,"data_stored":120626523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2477,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001396"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29936,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513045+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513045+0000","last_peered":"2024-09-05T13:25:28.513045+0000","last_clean":"2024-09-05T13:25:28.513045+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513045+0000","last_undegraded":"2024-09-05T13:25:28.513045+0000","last_fullsized":"2024-09-05T13:25:28.513045+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":22123,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:31.225993+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:31.225993+0000","last_peered":"2024-09-05T13:25:31.225993+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:31.225993+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45157,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429984,"kb_used_data":46932,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93941856,"statfs":{"total":96636764160,"available":96196460544,"internally_reserved":0,"allocated":48058368,"data_stored":47710691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515364,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856476,"statfs":{"total":96636764160,"available":96109031424,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:32.820 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:33.201 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:33.214 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-05T13:25:31.778505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45158,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507884,"kb_used_data":118840,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281607636,"statfs":{"total":289910292480,"available":288366219264,"internally_reserved":0,"allocated":121692160,"data_stored":120626523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2477,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001396"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29936,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513045+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513045+0000","last_peered":"2024-09-05T13:25:28.513045+0000","last_clean":"2024-09-05T13:25:28.513045+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513045+0000","last_undegraded":"2024-09-05T13:25:28.513045+0000","last_fullsized":"2024-09-05T13:25:28.513045+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":22123,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:31.225993+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:31.225993+0000","last_peered":"2024-09-05T13:25:31.225993+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:31.225993+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45157,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429984,"kb_used_data":46932,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93941856,"statfs":{"total":96636764160,"available":96196460544,"internally_reserved":0,"allocated":48058368,"data_stored":47710691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515364,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856476,"statfs":{"total":96636764160,"available":96109031424,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:33.215 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:33.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21333, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:28.510489+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:28.510489+0000', 'last_peered': '2024-09-05T13:25:28.510489+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:28.510489+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:33.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:33.216 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 22123, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:31.225993+0000', 'last_change': '2024-09-05T13:25:27.154156+0000', 'last_active': '2024-09-05T13:25:31.225993+0000', 'last_peered': '2024-09-05T13:25:31.225993+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:31.225993+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3888, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 532, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3888}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:33.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:33.624 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:33.624 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8341/6 objects degraded (139016.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":860,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1544073216,"bytes_avail":288366219264,"bytes_total":289910292480,"degraded_objects":8341,"degraded_total":6,"degraded_ratio":1390.166666666667,"recovering_objects_per_sec":309,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2477,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:33.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:33.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:33.939 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:33.939 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:33.954 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-05T13:25:33.778820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45158,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507884,"kb_used_data":118840,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281607636,"statfs":{"total":289910292480,"available":288366219264,"internally_reserved":0,"allocated":121692160,"data_stored":120626523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2477,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001390"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29936,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513045+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513045+0000","last_peered":"2024-09-05T13:25:28.513045+0000","last_clean":"2024-09-05T13:25:28.513045+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513045+0000","last_undegraded":"2024-09-05T13:25:28.513045+0000","last_fullsized":"2024-09-05T13:25:28.513045+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":22123,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:31.225993+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:31.225993+0000","last_peered":"2024-09-05T13:25:31.225993+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:31.225993+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45157,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429984,"kb_used_data":46932,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93941856,"statfs":{"total":96636764160,"available":96196460544,"internally_reserved":0,"allocated":48058368,"data_stored":47710691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515364,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856476,"statfs":{"total":96636764160,"available":96109031424,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:36.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:37.269 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:37.270 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:37.282 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-05T13:25:35.779109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45158,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496824,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281618696,"statfs":{"total":289910292480,"available":288377544704,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":910,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.288613"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29936,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513045+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513045+0000","last_peered":"2024-09-05T13:25:28.513045+0000","last_clean":"2024-09-05T13:25:28.513045+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513045+0000","last_undegraded":"2024-09-05T13:25:28.513045+0000","last_fullsized":"2024-09-05T13:25:28.513045+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":22123,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:31.225993+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:31.225993+0000","last_peered":"2024-09-05T13:25:31.225993+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:31.225993+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45157,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515364,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856476,"statfs":{"total":96636764160,"available":96109031424,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:37.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:37.581 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:37.596 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-05T13:25:35.779109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45158,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496824,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281618696,"statfs":{"total":289910292480,"available":288377544704,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":910,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.288613"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29936,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513045+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513045+0000","last_peered":"2024-09-05T13:25:28.513045+0000","last_clean":"2024-09-05T13:25:28.513045+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513045+0000","last_undegraded":"2024-09-05T13:25:28.513045+0000","last_fullsized":"2024-09-05T13:25:28.513045+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":22123,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:31.225993+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:31.225993+0000","last_peered":"2024-09-05T13:25:31.225993+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:31.225993+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45157,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515364,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856476,"statfs":{"total":96636764160,"available":96109031424,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:37.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:37.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21333, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:28.510489+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:28.510489+0000', 'last_peered': '2024-09-05T13:25:28.510489+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:28.510489+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:37.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:37.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 22123, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:31.225993+0000', 'last_change': '2024-09-05T13:25:27.154156+0000', 'last_active': '2024-09-05T13:25:31.225993+0000', 'last_peered': '2024-09-05T13:25:31.225993+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:31.225993+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3888, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 532, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3888}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:37.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:37.988 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:37.988 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7808/6 objects degraded (130133.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":864,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1537003520,"bytes_avail":288373288960,"bytes_total":289910292480,"degraded_objects":7808,"degraded_total":6,"degraded_ratio":1301.3333333333333,"recovering_objects_per_sec":144,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1443,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:38.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:38.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:38.305 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:38.306 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:38.319 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-05T13:25:37.779404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45691,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500980,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281614540,"statfs":{"total":289910292480,"available":288373288960,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1443,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1443,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001599"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":23189,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:36.226122+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:36.226122+0000","last_peered":"2024-09-05T13:25:36.226122+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:36.226122+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3355}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45690,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519520,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852320,"statfs":{"total":96636764160,"available":96104775680,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:41.321 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:41.651 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:41.652 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:41.663 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-09-05T13:25:39.779772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45691,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500980,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281614540,"statfs":{"total":289910292480,"available":288373288960,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1443,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1443,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.274942"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":23189,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:36.226122+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:36.226122+0000","last_peered":"2024-09-05T13:25:36.226122+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:36.226122+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3355}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45690,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519520,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852320,"statfs":{"total":96636764160,"available":96104775680,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:41.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:41.980 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:41.980 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:41.992 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-05T13:25:41.780140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46288,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505140,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281610380,"statfs":{"total":289910292480,"available":288369029120,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2040,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001996"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":24383,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:41.228606+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:41.228606+0000","last_peered":"2024-09-05T13:25:41.228606+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:41.228606+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46287,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523680,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848160,"statfs":{"total":96636764160,"available":96100515840,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:41.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:41.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21333, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:28.510489+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:28.510489+0000', 'last_peered': '2024-09-05T13:25:28.510489+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:28.510489+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:41.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:41.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 24383, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:41.228606+0000', 'last_change': '2024-09-05T13:25:27.154156+0000', 'last_active': '2024-09-05T13:25:41.228606+0000', 'last_peered': '2024-09-05T13:25:41.228606+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:41.228606+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2758, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1662, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2758}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:41.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:42.363 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:42.363 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7211/6 objects degraded (120183.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":869,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1541263360,"bytes_avail":288369029120,"bytes_total":289910292480,"degraded_objects":7211,"degraded_total":6,"degraded_ratio":1201.8333333333333,"recovering_objects_per_sec":169,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2040,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:42.375 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:42.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:42.684 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:42.684 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:42.696 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-05T13:25:41.780140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46288,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505140,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281610380,"statfs":{"total":289910292480,"available":288369029120,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2040,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001996"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":24383,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:41.228606+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:41.228606+0000","last_peered":"2024-09-05T13:25:41.228606+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:41.228606+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46287,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523680,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848160,"statfs":{"total":96636764160,"available":96100515840,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:45.699 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:46.001 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:46.031 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:46.032 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-05T13:25:45.780773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46288,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505140,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281610380,"statfs":{"total":289910292480,"available":288369029120,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1130,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001953"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":24383,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:41.228606+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:41.228606+0000","last_peered":"2024-09-05T13:25:41.228606+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:41.228606+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46287,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523680,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848160,"statfs":{"total":96636764160,"available":96100515840,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:46.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:46.331 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:46.343 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-05T13:25:45.780773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46288,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505140,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281610380,"statfs":{"total":289910292480,"available":288369029120,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1130,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001953"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":24383,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:41.228606+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:41.228606+0000","last_peered":"2024-09-05T13:25:41.228606+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:41.228606+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46287,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523680,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848160,"statfs":{"total":96636764160,"available":96100515840,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:46.344 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:46.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21333, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:28.510489+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:28.510489+0000', 'last_peered': '2024-09-05T13:25:28.510489+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:28.510489+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:46.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:46.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 24383, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:41.228606+0000', 'last_change': '2024-09-05T13:25:27.154156+0000', 'last_active': '2024-09-05T13:25:41.228606+0000', 'last_peered': '2024-09-05T13:25:41.228606+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:41.228606+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2758, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1662, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2758}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:46.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:46.715 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:46.715 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7211/6 objects degraded (120183.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":873,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1541263360,"bytes_avail":288369029120,"bytes_total":289910292480,"degraded_objects":7211,"degraded_total":6,"degraded_ratio":1201.8333333333333,"recovering_objects_per_sec":94,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1130,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:46.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:46.728 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:47.027 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:47.028 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:47.040 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-05T13:25:45.780773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46288,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505140,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281610380,"statfs":{"total":289910292480,"available":288369029120,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1130,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001953"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20898,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508146+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508146+0000","last_peered":"2024-09-05T13:25:28.508146+0000","last_clean":"2024-09-05T13:25:28.508146+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508146+0000","last_undegraded":"2024-09-05T13:25:28.508146+0000","last_fullsized":"2024-09-05T13:25:28.508146+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507742+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507742+0000","last_peered":"2024-09-05T13:25:28.507742+0000","last_clean":"2024-09-05T13:25:28.507742+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507742+0000","last_undegraded":"2024-09-05T13:25:28.507742+0000","last_fullsized":"2024-09-05T13:25:28.507742+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21266,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510272+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510272+0000","last_peered":"2024-09-05T13:25:28.510272+0000","last_clean":"2024-09-05T13:25:28.510272+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510272+0000","last_undegraded":"2024-09-05T13:25:28.510272+0000","last_fullsized":"2024-09-05T13:25:28.510272+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":24383,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:41.228606+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:41.228606+0000","last_peered":"2024-09-05T13:25:41.228606+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:41.228606+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46287,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523680,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848160,"statfs":{"total":96636764160,"available":96100515840,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:50.042 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:50.348 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:50.349 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:50.361 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-05T13:25:49.781443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46960,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496820,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281618700,"statfs":{"total":289910292480,"available":288377548800,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002038"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20899,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508186+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508186+0000","last_peered":"2024-09-05T13:25:28.508186+0000","last_clean":"2024-09-05T13:25:28.508186+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508186+0000","last_undegraded":"2024-09-05T13:25:28.508186+0000","last_fullsized":"2024-09-05T13:25:28.508186+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507783+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507783+0000","last_peered":"2024-09-05T13:25:28.507783+0000","last_clean":"2024-09-05T13:25:28.507783+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507783+0000","last_undegraded":"2024-09-05T13:25:28.507783+0000","last_fullsized":"2024-09-05T13:25:28.507783+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":25727,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:46.227020+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:46.227020+0000","last_peered":"2024-09-05T13:25:46.227020+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:46.227020+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46959,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515360,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856480,"statfs":{"total":96636764160,"available":96109035520,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:50.363 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:50.665 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:50.665 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:50.678 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-05T13:25:49.781443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46960,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496820,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281618700,"statfs":{"total":289910292480,"available":288377548800,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002038"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20899,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508186+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508186+0000","last_peered":"2024-09-05T13:25:28.508186+0000","last_clean":"2024-09-05T13:25:28.508186+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508186+0000","last_undegraded":"2024-09-05T13:25:28.508186+0000","last_fullsized":"2024-09-05T13:25:28.508186+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507783+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507783+0000","last_peered":"2024-09-05T13:25:28.507783+0000","last_clean":"2024-09-05T13:25:28.507783+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507783+0000","last_undegraded":"2024-09-05T13:25:28.507783+0000","last_fullsized":"2024-09-05T13:25:28.507783+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":25727,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:46.227020+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:46.227020+0000","last_peered":"2024-09-05T13:25:46.227020+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:46.227020+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46959,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515360,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856480,"statfs":{"total":96636764160,"available":96109035520,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:50.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:50.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21333, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:28.510489+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:28.510489+0000', 'last_peered': '2024-09-05T13:25:28.510489+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:28.510489+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:50.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:50.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 25727, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:46.227020+0000', 'last_change': '2024-09-05T13:25:27.154156+0000', 'last_active': '2024-09-05T13:25:46.227020+0000', 'last_peered': '2024-09-05T13:25:46.227020+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:46.227020+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2086, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2334, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:50.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:51.094 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:51.094 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6539/6 objects degraded (108983.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":877,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1532743680,"bytes_avail":288377548800,"bytes_total":289910292480,"degraded_objects":6539,"degraded_total":6,"degraded_ratio":1089.8333333333333,"recovering_objects_per_sec":105,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1269,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:51.106 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:51.107 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:51.414 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:51.414 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:51.426 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-05T13:25:49.781443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46960,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496820,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281618700,"statfs":{"total":289910292480,"available":288377548800,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002038"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20899,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508186+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508186+0000","last_peered":"2024-09-05T13:25:28.508186+0000","last_clean":"2024-09-05T13:25:28.508186+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508186+0000","last_undegraded":"2024-09-05T13:25:28.508186+0000","last_fullsized":"2024-09-05T13:25:28.508186+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507783+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507783+0000","last_peered":"2024-09-05T13:25:28.507783+0000","last_clean":"2024-09-05T13:25:28.507783+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507783+0000","last_undegraded":"2024-09-05T13:25:28.507783+0000","last_fullsized":"2024-09-05T13:25:28.507783+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510240+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510240+0000","last_peered":"2024-09-05T13:25:28.510240+0000","last_clean":"2024-09-05T13:25:28.510240+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510240+0000","last_undegraded":"2024-09-05T13:25:28.510240+0000","last_fullsized":"2024-09-05T13:25:28.510240+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":25727,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:46.227020+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:46.227020+0000","last_peered":"2024-09-05T13:25:46.227020+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:46.227020+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510314+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510314+0000","last_peered":"2024-09-05T13:25:28.510314+0000","last_clean":"2024-09-05T13:25:28.510314+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510314+0000","last_undegraded":"2024-09-05T13:25:28.510314+0000","last_fullsized":"2024-09-05T13:25:28.510314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46959,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515360,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856480,"statfs":{"total":96636764160,"available":96109035520,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:54.428 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:54.739 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:54.740 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:54.752 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-05T13:25:53.782092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47783,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500980,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281614540,"statfs":{"total":289910292480,"available":288373288960,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1495,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001951"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20899,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508186+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508186+0000","last_peered":"2024-09-05T13:25:28.508186+0000","last_clean":"2024-09-05T13:25:28.508186+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508186+0000","last_undegraded":"2024-09-05T13:25:28.508186+0000","last_fullsized":"2024-09-05T13:25:28.508186+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507783+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507783+0000","last_peered":"2024-09-05T13:25:28.507783+0000","last_clean":"2024-09-05T13:25:28.507783+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507783+0000","last_undegraded":"2024-09-05T13:25:28.507783+0000","last_fullsized":"2024-09-05T13:25:28.507783+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510289+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510289+0000","last_peered":"2024-09-05T13:25:28.510289+0000","last_clean":"2024-09-05T13:25:28.510289+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510289+0000","last_undegraded":"2024-09-05T13:25:28.510289+0000","last_fullsized":"2024-09-05T13:25:28.510289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":27373,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:51.228881+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:51.228881+0000","last_peered":"2024-09-05T13:25:51.228881+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:51.228881+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510352+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510352+0000","last_peered":"2024-09-05T13:25:28.510352+0000","last_clean":"2024-09-05T13:25:28.510352+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510352+0000","last_undegraded":"2024-09-05T13:25:28.510352+0000","last_fullsized":"2024-09-05T13:25:28.510352+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47782,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519520,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852320,"statfs":{"total":96636764160,"available":96104775680,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:54.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:55.066 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:55.066 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:55.078 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-05T13:25:53.782092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47783,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500980,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281614540,"statfs":{"total":289910292480,"available":288373288960,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1495,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001951"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20899,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508186+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508186+0000","last_peered":"2024-09-05T13:25:28.508186+0000","last_clean":"2024-09-05T13:25:28.508186+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508186+0000","last_undegraded":"2024-09-05T13:25:28.508186+0000","last_fullsized":"2024-09-05T13:25:28.508186+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507783+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507783+0000","last_peered":"2024-09-05T13:25:28.507783+0000","last_clean":"2024-09-05T13:25:28.507783+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507783+0000","last_undegraded":"2024-09-05T13:25:28.507783+0000","last_fullsized":"2024-09-05T13:25:28.507783+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510289+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510289+0000","last_peered":"2024-09-05T13:25:28.510289+0000","last_clean":"2024-09-05T13:25:28.510289+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510289+0000","last_undegraded":"2024-09-05T13:25:28.510289+0000","last_fullsized":"2024-09-05T13:25:28.510289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":27373,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:51.228881+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:51.228881+0000","last_peered":"2024-09-05T13:25:51.228881+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:51.228881+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510352+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510352+0000","last_peered":"2024-09-05T13:25:28.510352+0000","last_clean":"2024-09-05T13:25:28.510352+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510352+0000","last_undegraded":"2024-09-05T13:25:28.510352+0000","last_fullsized":"2024-09-05T13:25:28.510352+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47782,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519520,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852320,"statfs":{"total":96636764160,"available":96104775680,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:55.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:55.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21333, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:28.510489+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:28.510489+0000', 'last_peered': '2024-09-05T13:25:28.510489+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:28.510489+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:55.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:55.080 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 27373, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:51.228881+0000', 'last_change': '2024-09-05T13:25:27.154156+0000', 'last_active': '2024-09-05T13:25:51.228881+0000', 'last_peered': '2024-09-05T13:25:51.228881+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:51.228881+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1263, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3157, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1263}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:55.081 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:55.469 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:55.470 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5716/6 objects degraded (95266.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":882,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1537003520,"bytes_avail":288373288960,"bytes_total":289910292480,"degraded_objects":5716,"degraded_total":6,"degraded_ratio":952.66666666666652,"recovering_objects_per_sec":124,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1495,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:55.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:55.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:55.809 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:55.809 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:55.822 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-05T13:25:55.782426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47783,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500980,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281614540,"statfs":{"total":289910292480,"available":288373288960,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1495,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1495,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001932"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20899,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508186+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508186+0000","last_peered":"2024-09-05T13:25:28.508186+0000","last_clean":"2024-09-05T13:25:28.508186+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508186+0000","last_undegraded":"2024-09-05T13:25:28.508186+0000","last_fullsized":"2024-09-05T13:25:28.508186+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507783+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507783+0000","last_peered":"2024-09-05T13:25:28.507783+0000","last_clean":"2024-09-05T13:25:28.507783+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507783+0000","last_undegraded":"2024-09-05T13:25:28.507783+0000","last_fullsized":"2024-09-05T13:25:28.507783+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510289+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510289+0000","last_peered":"2024-09-05T13:25:28.510289+0000","last_clean":"2024-09-05T13:25:28.510289+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510289+0000","last_undegraded":"2024-09-05T13:25:28.510289+0000","last_fullsized":"2024-09-05T13:25:28.510289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":27373,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:51.228881+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:51.228881+0000","last_peered":"2024-09-05T13:25:51.228881+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:51.228881+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510352+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510352+0000","last_peered":"2024-09-05T13:25:28.510352+0000","last_clean":"2024-09-05T13:25:28.510352+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510352+0000","last_undegraded":"2024-09-05T13:25:28.510352+0000","last_fullsized":"2024-09-05T13:25:28.510352+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47782,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519520,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852320,"statfs":{"total":96636764160,"available":96104775680,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:58.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:59.141 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:59.142 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:59.153 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-05T13:25:57.876704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49012,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513460,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1405491,"kb_avail":281602060,"statfs":{"total":289910292480,"available":288360509440,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1439222790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2052,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.095261"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20899,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508186+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508186+0000","last_peered":"2024-09-05T13:25:28.508186+0000","last_clean":"2024-09-05T13:25:28.508186+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508186+0000","last_undegraded":"2024-09-05T13:25:28.508186+0000","last_fullsized":"2024-09-05T13:25:28.508186+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507783+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507783+0000","last_peered":"2024-09-05T13:25:28.507783+0000","last_clean":"2024-09-05T13:25:28.507783+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507783+0000","last_undegraded":"2024-09-05T13:25:28.507783+0000","last_fullsized":"2024-09-05T13:25:28.507783+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510289+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510289+0000","last_peered":"2024-09-05T13:25:28.510289+0000","last_clean":"2024-09-05T13:25:28.510289+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510289+0000","last_undegraded":"2024-09-05T13:25:28.510289+0000","last_fullsized":"2024-09-05T13:25:28.510289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":29831,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:56.231672+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:56.231672+0000","last_peered":"2024-09-05T13:25:56.231672+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:56.231672+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510352+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510352+0000","last_peered":"2024-09-05T13:25:28.510352+0000","last_clean":"2024-09-05T13:25:28.510352+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510352+0000","last_undegraded":"2024-09-05T13:25:28.510352+0000","last_fullsized":"2024-09-05T13:25:28.510352+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49011,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532000,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":531451,"kb_avail":93839840,"statfs":{"total":96636764160,"available":96091996160,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":544206511},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:59.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:25:59.464 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:59.464 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:25:59.476 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-05T13:25:57.876704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49012,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513460,"kb_used_data":107900,"kb_used_omap":12,"kb_used_meta":1405491,"kb_avail":281602060,"statfs":{"total":289910292480,"available":288360509440,"internally_reserved":0,"allocated":110489600,"data_stored":109423963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1439222790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2052,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.095261"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20899,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.508186+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:28.508186+0000","last_peered":"2024-09-05T13:25:28.508186+0000","last_clean":"2024-09-05T13:25:28.508186+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:28.508186+0000","last_undegraded":"2024-09-05T13:25:28.508186+0000","last_fullsized":"2024-09-05T13:25:28.508186+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507783+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:28.507783+0000","last_peered":"2024-09-05T13:25:28.507783+0000","last_clean":"2024-09-05T13:25:28.507783+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:28.507783+0000","last_undegraded":"2024-09-05T13:25:28.507783+0000","last_fullsized":"2024-09-05T13:25:28.507783+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507531+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:28.507531+0000","last_peered":"2024-09-05T13:25:28.507531+0000","last_clean":"2024-09-05T13:25:28.507531+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:28.507531+0000","last_undegraded":"2024-09-05T13:25:28.507531+0000","last_fullsized":"2024-09-05T13:25:28.507531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507581+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:28.507581+0000","last_peered":"2024-09-05T13:25:28.507581+0000","last_clean":"2024-09-05T13:25:28.507581+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:28.507581+0000","last_undegraded":"2024-09-05T13:25:28.507581+0000","last_fullsized":"2024-09-05T13:25:28.507581+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510289+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510289+0000","last_peered":"2024-09-05T13:25:28.510289+0000","last_clean":"2024-09-05T13:25:28.510289+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510289+0000","last_undegraded":"2024-09-05T13:25:28.510289+0000","last_fullsized":"2024-09-05T13:25:28.510289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29531,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507287+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:28.507287+0000","last_peered":"2024-09-05T13:25:28.507287+0000","last_clean":"2024-09-05T13:25:28.507287+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:28.507287+0000","last_undegraded":"2024-09-05T13:25:28.507287+0000","last_fullsized":"2024-09-05T13:25:28.507287+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":29831,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:56.231672+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:56.231672+0000","last_peered":"2024-09-05T13:25:56.231672+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:56.231672+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510352+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510352+0000","last_peered":"2024-09-05T13:25:28.510352+0000","last_clean":"2024-09-05T13:25:28.510352+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510352+0000","last_undegraded":"2024-09-05T13:25:28.510352+0000","last_fullsized":"2024-09-05T13:25:28.510352+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.507902+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:28.507902+0000","last_peered":"2024-09-05T13:25:28.507902+0000","last_clean":"2024-09-05T13:25:28.507902+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:28.507902+0000","last_undegraded":"2024-09-05T13:25:28.507902+0000","last_fullsized":"2024-09-05T13:25:28.507902+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49011,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":108908544,"data_stored":108894092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418924,"kb_used_data":35992,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93952916,"statfs":{"total":96636764160,"available":96207785984,"internally_reserved":0,"allocated":36855808,"data_stored":36508131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532000,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":531451,"kb_avail":93839840,"statfs":{"total":96636764160,"available":96091996160,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":544206511},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36331520,"data_stored":36331520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:25:59.477 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:25:59.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 21333, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:28.510489+0000', 'last_change': '2024-09-05T13:13:18.569003+0000', 'last_active': '2024-09-05T13:25:28.510489+0000', 'last_peered': '2024-09-05T13:25:28.510489+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:25:28.510489+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4453, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:59.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-05T13:25:59.478 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'20878", 'reported_seq': 29831, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:25:56.231672+0000', 'last_change': '2024-09-05T13:25:27.154156+0000', 'last_active': '2024-09-05T13:25:56.231672+0000', 'last_peered': '2024-09-05T13:25:56.231672+0000', 'last_clean': '2024-09-05T13:13:15.895786+0000', 'last_became_active': '2024-09-05T13:13:18.548471+0000', 'last_became_peered': '2024-09-05T13:13:18.548471+0000', 'last_unstale': '2024-09-05T13:25:56.231672+0000', 'last_undegraded': '2024-09-05T13:13:18.008473+0000', 'last_fullsized': '2024-09-05T13:13:17.904034+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3905", 'last_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:06.992813+0000', 'objects_scrubbed': 3847, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1924, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 34, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17029, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4386, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -34}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:25:59.478 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:25:59.878 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:25:59.878 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4487/6 objects degraded (74783.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":886,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1549783040,"bytes_avail":288360509440,"bytes_total":289910292480,"degraded_objects":4487,"degraded_total":6,"degraded_ratio":747.83333333333348,"recovering_objects_per_sec":205,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2052,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:25:59.892 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:25:59.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:00.195 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:00.195 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:00.209 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-05T13:25:59.783104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49012,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495796,"kb_used_data":90240,"kb_used_omap":12,"kb_used_meta":1405491,"kb_avail":281619724,"statfs":{"total":289910292480,"available":288378597376,"internally_reserved":0,"allocated":92405760,"data_stored":91329228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1439222790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1229,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1229,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001361"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29937,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.513091+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:28.513091+0000","last_peered":"2024-09-05T13:25:28.513091+0000","last_clean":"2024-09-05T13:25:28.513091+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:28.513091+0000","last_undegraded":"2024-09-05T13:25:28.513091+0000","last_fullsized":"2024-09-05T13:25:28.513091+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28614,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510337+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:28.510337+0000","last_peered":"2024-09-05T13:25:28.510337+0000","last_clean":"2024-09-05T13:25:28.510337+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:28.510337+0000","last_undegraded":"2024-09-05T13:25:28.510337+0000","last_fullsized":"2024-09-05T13:25:28.510337+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21057,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509918+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:28.509918+0000","last_peered":"2024-09-05T13:25:28.509918+0000","last_clean":"2024-09-05T13:25:28.509918+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:28.509918+0000","last_undegraded":"2024-09-05T13:25:28.509918+0000","last_fullsized":"2024-09-05T13:25:28.509918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21333,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:28.510489+0000","last_change":"2024-09-05T13:13:18.569003+0000","last_active":"2024-09-05T13:25:28.510489+0000","last_peered":"2024-09-05T13:25:28.510489+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:25:28.510489+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29694,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510173+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:28.510173+0000","last_peered":"2024-09-05T13:25:28.510173+0000","last_clean":"2024-09-05T13:25:28.510173+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:28.510173+0000","last_undegraded":"2024-09-05T13:25:28.510173+0000","last_fullsized":"2024-09-05T13:25:28.510173+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21276,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.509713+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:28.509713+0000","last_peered":"2024-09-05T13:25:28.509713+0000","last_clean":"2024-09-05T13:25:28.509713+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:28.509713+0000","last_undegraded":"2024-09-05T13:25:28.509713+0000","last_fullsized":"2024-09-05T13:25:28.509713+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510115+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:28.510115+0000","last_peered":"2024-09-05T13:25:28.510115+0000","last_clean":"2024-09-05T13:25:28.510115+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:28.510115+0000","last_undegraded":"2024-09-05T13:25:28.510115+0000","last_fullsized":"2024-09-05T13:25:28.510115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21267,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510311+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:28.510311+0000","last_peered":"2024-09-05T13:25:28.510311+0000","last_clean":"2024-09-05T13:25:28.510311+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:28.510311+0000","last_undegraded":"2024-09-05T13:25:28.510311+0000","last_fullsized":"2024-09-05T13:25:28.510311+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30008,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510688+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:28.510688+0000","last_peered":"2024-09-05T13:25:28.510688+0000","last_clean":"2024-09-05T13:25:28.510688+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:28.510688+0000","last_undegraded":"2024-09-05T13:25:28.510688+0000","last_fullsized":"2024-09-05T13:25:28.510688+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510387+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:28.510387+0000","last_peered":"2024-09-05T13:25:28.510387+0000","last_clean":"2024-09-05T13:25:28.510387+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:28.510387+0000","last_undegraded":"2024-09-05T13:25:28.510387+0000","last_fullsized":"2024-09-05T13:25:28.510387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510289+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:28.510289+0000","last_peered":"2024-09-05T13:25:28.510289+0000","last_clean":"2024-09-05T13:25:28.510289+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:28.510289+0000","last_undegraded":"2024-09-05T13:25:28.510289+0000","last_fullsized":"2024-09-05T13:25:28.510289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":29831,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:25:56.231672+0000","last_change":"2024-09-05T13:25:27.154156+0000","last_active":"2024-09-05T13:25:56.231672+0000","last_peered":"2024-09-05T13:25:56.231672+0000","last_clean":"2024-09-05T13:13:15.895786+0000","last_became_active":"2024-09-05T13:13:18.548471+0000","last_became_peered":"2024-09-05T13:13:18.548471+0000","last_unstale":"2024-09-05T13:25:56.231672+0000","last_undegraded":"2024-09-05T13:13:18.008473+0000","last_fullsized":"2024-09-05T13:13:17.904034+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-34}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510352+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:28.510352+0000","last_peered":"2024-09-05T13:25:28.510352+0000","last_clean":"2024-09-05T13:25:28.510352+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:28.510352+0000","last_undegraded":"2024-09-05T13:25:28.510352+0000","last_fullsized":"2024-09-05T13:25:28.510352+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21311,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.585442+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:28.585442+0000","last_peered":"2024-09-05T13:25:28.585442+0000","last_clean":"2024-09-05T13:25:28.585442+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:28.585442+0000","last_undegraded":"2024-09-05T13:25:28.585442+0000","last_fullsized":"2024-09-05T13:25:28.585442+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21264,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510442+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:28.510442+0000","last_peered":"2024-09-05T13:25:28.510442+0000","last_clean":"2024-09-05T13:25:28.510442+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:28.510442+0000","last_undegraded":"2024-09-05T13:25:28.510442+0000","last_fullsized":"2024-09-05T13:25:28.510442+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510497+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:28.510497+0000","last_peered":"2024-09-05T13:25:28.510497+0000","last_clean":"2024-09-05T13:25:28.510497+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:28.510497+0000","last_undegraded":"2024-09-05T13:25:28.510497+0000","last_fullsized":"2024-09-05T13:25:28.510497+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510382+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:28.510382+0000","last_peered":"2024-09-05T13:25:28.510382+0000","last_clean":"2024-09-05T13:25:28.510382+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:28.510382+0000","last_undegraded":"2024-09-05T13:25:28.510382+0000","last_fullsized":"2024-09-05T13:25:28.510382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21356,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-05T13:25:28.510242+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:28.510242+0000","last_peered":"2024-09-05T13:25:28.510242+0000","last_clean":"2024-09-05T13:25:28.510242+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:28.510242+0000","last_undegraded":"2024-09-05T13:25:28.510242+0000","last_fullsized":"2024-09-05T13:25:28.510242+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49011,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90808320,"data_stored":90793868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562536,"kb_used_data":71380,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809304,"statfs":{"total":96636764160,"available":96060727296,"internally_reserved":0,"allocated":73093120,"data_stored":72737753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532000,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":531451,"kb_avail":93839840,"statfs":{"total":96636764160,"available":96091996160,"internally_reserved":0,"allocated":540672,"data_stored":178079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":544206511},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:03.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:03.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:03.531 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-05T13:26:01.783399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49545,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483344,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281632176,"statfs":{"total":289910292480,"available":288391348224,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001307"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":22336,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:01.225671+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:01.225671+0000","last_peered":"2024-09-05T13:26:01.225671+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:01.225671+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":499,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3954}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29698,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879740+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879740+0000","last_peered":"2024-09-05T13:25:57.879740+0000","last_clean":"2024-09-05T13:25:57.879740+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879740+0000","last_undegraded":"2024-09-05T13:25:57.879740+0000","last_fullsized":"2024-09-05T13:25:57.879740+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49544,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519528,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852312,"statfs":{"total":96636764160,"available":96104767488,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:03.532 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:03.869 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:03.870 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:03.884 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-09-05T13:26:03.783769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49545,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483344,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281632176,"statfs":{"total":289910292480,"available":288391348224,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001342"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":22336,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:01.225671+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:01.225671+0000","last_peered":"2024-09-05T13:26:01.225671+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:01.225671+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":499,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3954}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29698,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879740+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879740+0000","last_peered":"2024-09-05T13:25:57.879740+0000","last_clean":"2024-09-05T13:25:57.879740+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879740+0000","last_undegraded":"2024-09-05T13:25:57.879740+0000","last_fullsized":"2024-09-05T13:25:57.879740+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49544,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519528,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852312,"statfs":{"total":96636764160,"available":96104767488,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:03.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:26:03.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 22336, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:26:01.225671+0000', 'last_change': '2024-09-05T13:25:56.334055+0000', 'last_active': '2024-09-05T13:26:01.225671+0000', 'last_peered': '2024-09-05T13:26:01.225671+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:26:01.225671+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3954, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 499, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3954}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:26:03.885 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:26:04.275 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:04.275 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3954/6 objects degraded (65900.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":891,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1518944256,"bytes_avail":288391348224,"bytes_total":289910292480,"degraded_objects":3954,"degraded_total":6,"degraded_ratio":659,"recovering_objects_per_sec":220,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:26:04.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:26:04.289 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:04.587 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:04.588 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:04.601 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-09-05T13:26:03.783769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49545,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483344,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281632176,"statfs":{"total":289910292480,"available":288391348224,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001342"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":22336,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:01.225671+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:01.225671+0000","last_peered":"2024-09-05T13:26:01.225671+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:01.225671+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":499,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3954}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29698,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879740+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879740+0000","last_peered":"2024-09-05T13:25:57.879740+0000","last_clean":"2024-09-05T13:25:57.879740+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879740+0000","last_undegraded":"2024-09-05T13:25:57.879740+0000","last_fullsized":"2024-09-05T13:25:57.879740+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49544,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519528,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852312,"statfs":{"total":96636764160,"available":96104767488,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:07.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:07.957 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:07.969 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-09-05T13:26:07.784359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50100,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487504,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281628016,"statfs":{"total":289910292480,"available":288387088384,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1088,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001578"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":23447,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:06.229104+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:06.229104+0000","last_peered":"2024-09-05T13:26:06.229104+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:06.229104+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1054,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29698,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879740+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879740+0000","last_peered":"2024-09-05T13:25:57.879740+0000","last_clean":"2024-09-05T13:25:57.879740+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879740+0000","last_undegraded":"2024-09-05T13:25:57.879740+0000","last_fullsized":"2024-09-05T13:25:57.879740+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50099,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523688,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848152,"statfs":{"total":96636764160,"available":96100507648,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:07.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:08.284 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:08.284 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:08.296 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-09-05T13:26:07.784359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50100,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487504,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281628016,"statfs":{"total":289910292480,"available":288387088384,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1088,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001578"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":23447,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:06.229104+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:06.229104+0000","last_peered":"2024-09-05T13:26:06.229104+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:06.229104+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1054,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29698,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879740+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879740+0000","last_peered":"2024-09-05T13:25:57.879740+0000","last_clean":"2024-09-05T13:25:57.879740+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879740+0000","last_undegraded":"2024-09-05T13:25:57.879740+0000","last_fullsized":"2024-09-05T13:25:57.879740+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50099,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523688,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848152,"statfs":{"total":96636764160,"available":96100507648,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:08.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:26:08.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 23447, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:26:06.229104+0000', 'last_change': '2024-09-05T13:25:56.334055+0000', 'last_active': '2024-09-05T13:26:06.229104+0000', 'last_peered': '2024-09-05T13:26:06.229104+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:26:06.229104+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3399, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1054, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:26:08.298 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:26:08.664 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:08.664 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3399/6 objects degraded (56650.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":895,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1523204096,"bytes_avail":288387088384,"bytes_total":289910292480,"degraded_objects":3399,"degraded_total":6,"degraded_ratio":566.5,"recovering_objects_per_sec":108,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1088,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:26:08.679 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:26:08.679 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:08.989 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:08.990 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:09.002 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-09-05T13:26:07.784359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50100,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487504,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281628016,"statfs":{"total":289910292480,"available":288387088384,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1088,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001578"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":23447,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:06.229104+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:06.229104+0000","last_peered":"2024-09-05T13:26:06.229104+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:06.229104+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1054,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29698,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879740+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879740+0000","last_peered":"2024-09-05T13:25:57.879740+0000","last_clean":"2024-09-05T13:25:57.879740+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879740+0000","last_undegraded":"2024-09-05T13:25:57.879740+0000","last_fullsized":"2024-09-05T13:25:57.879740+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50099,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523688,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848152,"statfs":{"total":96636764160,"available":96100507648,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:12.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:12.309 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:12.309 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:12.319 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-05T13:26:11.784966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50708,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491664,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1401331,"kb_avail":281623856,"statfs":{"total":289910292480,"available":288382828544,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1434962950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1696,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001862"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":24663,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:11.230024+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:11.230024+0000","last_peered":"2024-09-05T13:26:11.230024+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:11.230024+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50707,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527848,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843992,"statfs":{"total":96636764160,"available":96096247808,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:12.321 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:12.639 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:12.639 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:12.651 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-05T13:26:11.784966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50708,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491664,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1401331,"kb_avail":281623856,"statfs":{"total":289910292480,"available":288382828544,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1434962950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1696,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001862"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":24663,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:11.230024+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:11.230024+0000","last_peered":"2024-09-05T13:26:11.230024+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:11.230024+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50707,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527848,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843992,"statfs":{"total":96636764160,"available":96096247808,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:12.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:26:12.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 24663, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:26:11.230024+0000', 'last_change': '2024-09-05T13:25:56.334055+0000', 'last_active': '2024-09-05T13:26:11.230024+0000', 'last_peered': '2024-09-05T13:26:11.230024+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:26:11.230024+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2791, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1662, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2791}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:26:12.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:26:13.080 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:13.080 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2791/6 objects degraded (46516.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":899,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1527463936,"bytes_avail":288382828544,"bytes_total":289910292480,"degraded_objects":2791,"degraded_total":6,"degraded_ratio":465.16666666666674,"recovering_objects_per_sec":141,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1696,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:26:13.094 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:26:13.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:13.413 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:13.414 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:13.428 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-05T13:26:11.784966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50708,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491664,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1401331,"kb_avail":281623856,"statfs":{"total":289910292480,"available":288382828544,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1434962950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1696,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001862"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":24663,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:11.230024+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:11.230024+0000","last_peered":"2024-09-05T13:26:11.230024+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:11.230024+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50707,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527848,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843992,"statfs":{"total":96636764160,"available":96096247808,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:16.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:16.744 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:16.744 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:16.757 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-05T13:26:15.785553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50708,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491664,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1401331,"kb_avail":281623856,"statfs":{"total":289910292480,"available":288382828544,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1434962950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001783"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":24663,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:11.230024+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:11.230024+0000","last_peered":"2024-09-05T13:26:11.230024+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:11.230024+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50707,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527848,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843992,"statfs":{"total":96636764160,"available":96096247808,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:16.758 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:17.062 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:17.063 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:17.077 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-05T13:26:15.785553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50708,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491664,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1401331,"kb_avail":281623856,"statfs":{"total":289910292480,"available":288382828544,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1434962950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1163,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1163,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001783"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":24663,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:11.230024+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:11.230024+0000","last_peered":"2024-09-05T13:26:11.230024+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:11.230024+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2791,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50707,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527848,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843992,"statfs":{"total":96636764160,"available":96096247808,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:26:17.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 24663, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:26:11.230024+0000', 'last_change': '2024-09-05T13:25:56.334055+0000', 'last_active': '2024-09-05T13:26:11.230024+0000', 'last_peered': '2024-09-05T13:26:11.230024+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:26:11.230024+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2791, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1662, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2791}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:26:17.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:26:17.488 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:17.488 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2791/6 objects degraded (46516.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":904,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1527463936,"bytes_avail":288382828544,"bytes_total":289910292480,"degraded_objects":2791,"degraded_total":6,"degraded_ratio":465.16666666666674,"recovering_objects_per_sec":96,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1163,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:26:17.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:26:17.502 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:17.819 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:17.820 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:17.833 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-05T13:26:17.785877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479184,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281636336,"statfs":{"total":289910292480,"available":288395608064,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001810"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":26031,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:16.234301+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:16.234301+0000","last_peered":"2024-09-05T13:26:16.234301+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:16.234301+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2346,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29535,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877823+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877823+0000","last_peered":"2024-09-05T13:25:57.877823+0000","last_clean":"2024-09-05T13:25:57.877823+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877823+0000","last_undegraded":"2024-09-05T13:25:57.877823+0000","last_fullsized":"2024-09-05T13:25:57.877823+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51391,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515368,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856472,"statfs":{"total":96636764160,"available":96109027328,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:20.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:21.149 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:21.164 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-05T13:26:19.786293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479184,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281636336,"statfs":{"total":289910292480,"available":288395608064,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001933"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":26031,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:16.234301+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:16.234301+0000","last_peered":"2024-09-05T13:26:16.234301+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:16.234301+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2346,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29536,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877885+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877885+0000","last_peered":"2024-09-05T13:25:57.877885+0000","last_clean":"2024-09-05T13:25:57.877885+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877885+0000","last_undegraded":"2024-09-05T13:25:57.877885+0000","last_fullsized":"2024-09-05T13:25:57.877885+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51391,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515368,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856472,"statfs":{"total":96636764160,"available":96109027328,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:21.166 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:21.475 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:21.489 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-05T13:26:19.786293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51392,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479184,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281636336,"statfs":{"total":289910292480,"available":288395608064,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001933"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21061,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878164+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878164+0000","last_peered":"2024-09-05T13:25:57.878164+0000","last_clean":"2024-09-05T13:25:57.878164+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878164+0000","last_undegraded":"2024-09-05T13:25:57.878164+0000","last_fullsized":"2024-09-05T13:25:57.878164+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":26031,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:16.234301+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:16.234301+0000","last_peered":"2024-09-05T13:26:16.234301+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:16.234301+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2346,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29536,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877885+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877885+0000","last_peered":"2024-09-05T13:25:57.877885+0000","last_clean":"2024-09-05T13:25:57.877885+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877885+0000","last_undegraded":"2024-09-05T13:25:57.877885+0000","last_fullsized":"2024-09-05T13:25:57.877885+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880125+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880125+0000","last_peered":"2024-09-05T13:25:57.880125+0000","last_clean":"2024-09-05T13:25:57.880125+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880125+0000","last_undegraded":"2024-09-05T13:25:57.880125+0000","last_fullsized":"2024-09-05T13:25:57.880125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51391,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515368,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856472,"statfs":{"total":96636764160,"available":96109027328,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:21.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:26:21.491 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 26031, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:26:16.234301+0000', 'last_change': '2024-09-05T13:25:56.334055+0000', 'last_active': '2024-09-05T13:26:16.234301+0000', 'last_peered': '2024-09-05T13:26:16.234301+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:26:16.234301+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2346, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:26:21.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:26:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:21.869 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2107/6 objects degraded (35116.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":908,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1514684416,"bytes_avail":288395608064,"bytes_total":289910292480,"degraded_objects":2107,"degraded_total":6,"degraded_ratio":351.16666666666674,"recovering_objects_per_sec":107,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1292,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:26:21.882 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:26:21.882 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:22.207 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-05T13:26:21.786602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52196,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487504,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281628016,"statfs":{"total":289910292480,"available":288387088384,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2096,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2096,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001925"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21062,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878217+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878217+0000","last_peered":"2024-09-05T13:25:57.878217+0000","last_clean":"2024-09-05T13:25:57.878217+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878217+0000","last_undegraded":"2024-09-05T13:25:57.878217+0000","last_fullsized":"2024-09-05T13:25:57.878217+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":27639,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:21.231240+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:21.231240+0000","last_peered":"2024-09-05T13:26:21.231240+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:21.231240+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29536,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877885+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877885+0000","last_peered":"2024-09-05T13:25:57.877885+0000","last_clean":"2024-09-05T13:25:57.877885+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877885+0000","last_undegraded":"2024-09-05T13:25:57.877885+0000","last_fullsized":"2024-09-05T13:25:57.877885+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880166+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880166+0000","last_peered":"2024-09-05T13:25:57.880166+0000","last_clean":"2024-09-05T13:25:57.880166+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880166+0000","last_undegraded":"2024-09-05T13:25:57.880166+0000","last_fullsized":"2024-09-05T13:25:57.880166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52195,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523688,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848152,"statfs":{"total":96636764160,"available":96100507648,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:25.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:25.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:25.519 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:25.531 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-05T13:26:23.786873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52196,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487504,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281628016,"statfs":{"total":289910292480,"available":288387088384,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001906"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21062,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878217+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878217+0000","last_peered":"2024-09-05T13:25:57.878217+0000","last_clean":"2024-09-05T13:25:57.878217+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878217+0000","last_undegraded":"2024-09-05T13:25:57.878217+0000","last_fullsized":"2024-09-05T13:25:57.878217+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":27639,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:21.231240+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:21.231240+0000","last_peered":"2024-09-05T13:26:21.231240+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:21.231240+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29536,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877885+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877885+0000","last_peered":"2024-09-05T13:25:57.877885+0000","last_clean":"2024-09-05T13:25:57.877885+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877885+0000","last_undegraded":"2024-09-05T13:25:57.877885+0000","last_fullsized":"2024-09-05T13:25:57.877885+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880166+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880166+0000","last_peered":"2024-09-05T13:25:57.880166+0000","last_clean":"2024-09-05T13:25:57.880166+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880166+0000","last_undegraded":"2024-09-05T13:25:57.880166+0000","last_fullsized":"2024-09-05T13:25:57.880166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52195,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523688,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848152,"statfs":{"total":96636764160,"available":96100507648,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:25.532 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:25.844 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:25.844 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:25.857 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-09-05T13:26:25.787158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52196,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487504,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281628016,"statfs":{"total":289910292480,"available":288387088384,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001883"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21062,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878217+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878217+0000","last_peered":"2024-09-05T13:25:57.878217+0000","last_clean":"2024-09-05T13:25:57.878217+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878217+0000","last_undegraded":"2024-09-05T13:25:57.878217+0000","last_fullsized":"2024-09-05T13:25:57.878217+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":27639,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:21.231240+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:21.231240+0000","last_peered":"2024-09-05T13:26:21.231240+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:21.231240+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29536,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877885+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877885+0000","last_peered":"2024-09-05T13:25:57.877885+0000","last_clean":"2024-09-05T13:25:57.877885+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877885+0000","last_undegraded":"2024-09-05T13:25:57.877885+0000","last_fullsized":"2024-09-05T13:25:57.877885+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880166+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880166+0000","last_peered":"2024-09-05T13:25:57.880166+0000","last_clean":"2024-09-05T13:25:57.880166+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880166+0000","last_undegraded":"2024-09-05T13:25:57.880166+0000","last_fullsized":"2024-09-05T13:25:57.880166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52195,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523688,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848152,"statfs":{"total":96636764160,"available":96100507648,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:25.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:26:25.858 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 27639, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:26:21.231240+0000', 'last_change': '2024-09-05T13:25:56.334055+0000', 'last_active': '2024-09-05T13:26:21.231240+0000', 'last_peered': '2024-09-05T13:26:21.231240+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:26:21.231240+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1303, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3150, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:26:25.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:26:26.259 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:26.259 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1303/6 objects degraded (21716.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":913,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1523204096,"bytes_avail":288387088384,"bytes_total":289910292480,"degraded_objects":1303,"degraded_total":6,"degraded_ratio":217.16666666666663,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1488,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:26:26.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:26:26.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:26.569 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:26.570 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:26.582 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-09-05T13:26:25.787158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52196,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487504,"kb_used_data":90264,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281628016,"statfs":{"total":289910292480,"available":288387088384,"internally_reserved":0,"allocated":92430336,"data_stored":91339240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001883"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20903,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878145+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:25:57.878145+0000","last_peered":"2024-09-05T13:25:57.878145+0000","last_clean":"2024-09-05T13:25:57.878145+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:25:57.878145+0000","last_undegraded":"2024-09-05T13:25:57.878145+0000","last_fullsized":"2024-09-05T13:25:57.878145+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21140,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878023+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:25:57.878023+0000","last_peered":"2024-09-05T13:25:57.878023+0000","last_clean":"2024-09-05T13:25:57.878023+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:25:57.878023+0000","last_undegraded":"2024-09-05T13:25:57.878023+0000","last_fullsized":"2024-09-05T13:25:57.878023+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21062,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878217+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878217+0000","last_peered":"2024-09-05T13:25:57.878217+0000","last_clean":"2024-09-05T13:25:57.878217+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878217+0000","last_undegraded":"2024-09-05T13:25:57.878217+0000","last_fullsized":"2024-09-05T13:25:57.878217+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":27639,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:21.231240+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:21.231240+0000","last_peered":"2024-09-05T13:26:21.231240+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:21.231240+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878228+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:25:57.878228+0000","last_peered":"2024-09-05T13:25:57.878228+0000","last_clean":"2024-09-05T13:25:57.878228+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:25:57.878228+0000","last_undegraded":"2024-09-05T13:25:57.878228+0000","last_fullsized":"2024-09-05T13:25:57.878228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29856,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878351+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:25:57.878351+0000","last_peered":"2024-09-05T13:25:57.878351+0000","last_clean":"2024-09-05T13:25:57.878351+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:25:57.878351+0000","last_undegraded":"2024-09-05T13:25:57.878351+0000","last_fullsized":"2024-09-05T13:25:57.878351+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29536,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.877885+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:25:57.877885+0000","last_peered":"2024-09-05T13:25:57.877885+0000","last_clean":"2024-09-05T13:25:57.877885+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:25:57.877885+0000","last_undegraded":"2024-09-05T13:25:57.877885+0000","last_fullsized":"2024-09-05T13:25:57.877885+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878272+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:25:57.878272+0000","last_peered":"2024-09-05T13:25:57.878272+0000","last_clean":"2024-09-05T13:25:57.878272+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:25:57.878272+0000","last_undegraded":"2024-09-05T13:25:57.878272+0000","last_fullsized":"2024-09-05T13:25:57.878272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880166+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880166+0000","last_peered":"2024-09-05T13:25:57.880166+0000","last_clean":"2024-09-05T13:25:57.880166+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880166+0000","last_undegraded":"2024-09-05T13:25:57.880166+0000","last_fullsized":"2024-09-05T13:25:57.880166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52195,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90800128,"data_stored":90792902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401260,"kb_used_data":18332,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93970580,"statfs":{"total":96636764160,"available":96225873920,"internally_reserved":0,"allocated":18771968,"data_stored":18413396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523688,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848152,"statfs":{"total":96636764160,"available":96100507648,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18231296,"data_stored":18231296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:29.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:29.877 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:29.877 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:29.888 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-09-05T13:26:29.788049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53344,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461396,"kb_used_data":72476,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281654124,"statfs":{"total":289910292480,"available":288413822976,"internally_reserved":0,"allocated":74215424,"data_stored":73113383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001446"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21062,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878217+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878217+0000","last_peered":"2024-09-05T13:25:57.878217+0000","last_clean":"2024-09-05T13:25:57.878217+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878217+0000","last_undegraded":"2024-09-05T13:25:57.878217+0000","last_fullsized":"2024-09-05T13:25:57.878217+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":29935,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:26.237213+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:26.237213+0000","last_peered":"2024-09-05T13:26:26.237213+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:26.237213+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880166+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880166+0000","last_peered":"2024-09-05T13:25:57.880166+0000","last_clean":"2024-09-05T13:25:57.880166+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880166+0000","last_undegraded":"2024-09-05T13:25:57.880166+0000","last_fullsized":"2024-09-05T13:25:57.880166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53343,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72568832,"data_stored":72561606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515368,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856472,"statfs":{"total":96636764160,"available":96109027328,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:29.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:30.170 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:30.170 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:30.182 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-09-05T13:26:29.788049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53344,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461396,"kb_used_data":72476,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281654124,"statfs":{"total":289910292480,"available":288413822976,"internally_reserved":0,"allocated":74215424,"data_stored":73113383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001446"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21062,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878217+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878217+0000","last_peered":"2024-09-05T13:25:57.878217+0000","last_clean":"2024-09-05T13:25:57.878217+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878217+0000","last_undegraded":"2024-09-05T13:25:57.878217+0000","last_fullsized":"2024-09-05T13:25:57.878217+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":29935,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:26.237213+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:26.237213+0000","last_peered":"2024-09-05T13:26:26.237213+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:26.237213+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880166+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880166+0000","last_peered":"2024-09-05T13:25:57.880166+0000","last_clean":"2024-09-05T13:25:57.880166+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880166+0000","last_undegraded":"2024-09-05T13:25:57.880166+0000","last_fullsized":"2024-09-05T13:25:57.880166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53343,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72568832,"data_stored":72561606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515368,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856472,"statfs":{"total":96636764160,"available":96109027328,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:30.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-05T13:26:30.183 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'21155", 'reported_seq': 29935, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-05T13:26:26.237213+0000', 'last_change': '2024-09-05T13:25:56.334055+0000', 'last_active': '2024-09-05T13:26:26.237213+0000', 'last_peered': '2024-09-05T13:26:26.237213+0000', 'last_clean': '2024-09-05T13:13:15.850773+0000', 'last_became_active': '2024-09-05T13:13:18.554314+0000', 'last_became_peered': '2024-09-05T13:13:18.554314+0000', 'last_unstale': '2024-09-05T13:26:26.237213+0000', 'last_undegraded': '2024-09-05T13:13:18.014991+0000', 'last_fullsized': '2024-09-05T13:13:17.903151+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3639", 'last_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-05T13:11:39.923202+0000', 'last_clean_scrub_stamp': '2024-09-05T13:13:02.937876+0000', 'objects_scrubbed': 3540, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1813, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17610, 'num_write_kb': 28140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4298, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -155}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-05T13:26:30.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-05T13:26:30.532 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:30.532 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 155/6 objects degraded (2583.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":917,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1725541995,"num_in_osds":3,"osd_in_since":1725541987,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":502,"bytes_used":1496469504,"bytes_avail":288413822976,"bytes_total":289910292480,"degraded_objects":155,"degraded_total":6,"degraded_ratio":25.833333333333329,"recovering_objects_per_sec":143,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1148,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-05T13:11:13:133060+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-05T13:14:15.539609+0000","services":{}},"progress_events":{}} 2024-09-05T13:26:30.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-05T13:26:30.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:30.823 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:30.824 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:30.836 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-09-05T13:26:29.788049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53344,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461396,"kb_used_data":72476,"kb_used_omap":12,"kb_used_meta":1388851,"kb_avail":281654124,"statfs":{"total":289910292480,"available":288413822976,"internally_reserved":0,"allocated":74215424,"data_stored":73113383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1422183430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1148,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001446"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29941,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878289+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:25:57.878289+0000","last_peered":"2024-09-05T13:25:57.878289+0000","last_clean":"2024-09-05T13:25:57.878289+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:25:57.878289+0000","last_undegraded":"2024-09-05T13:25:57.878289+0000","last_fullsized":"2024-09-05T13:25:57.878289+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28618,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879871+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:25:57.879871+0000","last_peered":"2024-09-05T13:25:57.879871+0000","last_clean":"2024-09-05T13:25:57.879871+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:25:57.879871+0000","last_undegraded":"2024-09-05T13:25:57.879871+0000","last_fullsized":"2024-09-05T13:25:57.879871+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21062,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878217+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:25:57.878217+0000","last_peered":"2024-09-05T13:25:57.878217+0000","last_clean":"2024-09-05T13:25:57.878217+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:25:57.878217+0000","last_undegraded":"2024-09-05T13:25:57.878217+0000","last_fullsized":"2024-09-05T13:25:57.878217+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":29935,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-05T13:26:26.237213+0000","last_change":"2024-09-05T13:25:56.334055+0000","last_active":"2024-09-05T13:26:26.237213+0000","last_peered":"2024-09-05T13:26:26.237213+0000","last_clean":"2024-09-05T13:13:15.850773+0000","last_became_active":"2024-09-05T13:13:18.554314+0000","last_became_peered":"2024-09-05T13:13:18.554314+0000","last_unstale":"2024-09-05T13:26:26.237213+0000","last_undegraded":"2024-09-05T13:13:18.014991+0000","last_fullsized":"2024-09-05T13:13:17.903151+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29699,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879785+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:25:57.879785+0000","last_peered":"2024-09-05T13:25:57.879785+0000","last_clean":"2024-09-05T13:25:57.879785+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:25:57.879785+0000","last_undegraded":"2024-09-05T13:25:57.879785+0000","last_fullsized":"2024-09-05T13:25:57.879785+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21280,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878760+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:25:57.878760+0000","last_peered":"2024-09-05T13:25:57.878760+0000","last_clean":"2024-09-05T13:25:57.878760+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:25:57.878760+0000","last_undegraded":"2024-09-05T13:25:57.878760+0000","last_fullsized":"2024-09-05T13:25:57.878760+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879717+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:25:57.879717+0000","last_peered":"2024-09-05T13:25:57.879717+0000","last_clean":"2024-09-05T13:25:57.879717+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:25:57.879717+0000","last_undegraded":"2024-09-05T13:25:57.879717+0000","last_fullsized":"2024-09-05T13:25:57.879717+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21271,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879879+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:25:57.879879+0000","last_peered":"2024-09-05T13:25:57.879879+0000","last_clean":"2024-09-05T13:25:57.879879+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:25:57.879879+0000","last_undegraded":"2024-09-05T13:25:57.879879+0000","last_fullsized":"2024-09-05T13:25:57.879879+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30012,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878471+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:25:57.878471+0000","last_peered":"2024-09-05T13:25:57.878471+0000","last_clean":"2024-09-05T13:25:57.878471+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:25:57.878471+0000","last_undegraded":"2024-09-05T13:25:57.878471+0000","last_fullsized":"2024-09-05T13:25:57.878471+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.878337+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:25:57.878337+0000","last_peered":"2024-09-05T13:25:57.878337+0000","last_clean":"2024-09-05T13:25:57.878337+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:25:57.878337+0000","last_undegraded":"2024-09-05T13:25:57.878337+0000","last_fullsized":"2024-09-05T13:25:57.878337+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879779+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:25:57.879779+0000","last_peered":"2024-09-05T13:25:57.879779+0000","last_clean":"2024-09-05T13:25:57.879779+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:25:57.879779+0000","last_undegraded":"2024-09-05T13:25:57.879779+0000","last_fullsized":"2024-09-05T13:25:57.879779+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21031,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.965120+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:25:57.965120+0000","last_peered":"2024-09-05T13:25:57.965120+0000","last_clean":"2024-09-05T13:25:57.965120+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:25:57.965120+0000","last_undegraded":"2024-09-05T13:25:57.965120+0000","last_fullsized":"2024-09-05T13:25:57.965120+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879822+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:25:57.879822+0000","last_peered":"2024-09-05T13:25:57.879822+0000","last_clean":"2024-09-05T13:25:57.879822+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:25:57.879822+0000","last_undegraded":"2024-09-05T13:25:57.879822+0000","last_fullsized":"2024-09-05T13:25:57.879822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21315,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879997+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:25:57.879997+0000","last_peered":"2024-09-05T13:25:57.879997+0000","last_clean":"2024-09-05T13:25:57.879997+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:25:57.879997+0000","last_undegraded":"2024-09-05T13:25:57.879997+0000","last_fullsized":"2024-09-05T13:25:57.879997+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21268,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879907+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:25:57.879907+0000","last_peered":"2024-09-05T13:25:57.879907+0000","last_clean":"2024-09-05T13:25:57.879907+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:25:57.879907+0000","last_undegraded":"2024-09-05T13:25:57.879907+0000","last_fullsized":"2024-09-05T13:25:57.879907+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880019+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:25:57.880019+0000","last_peered":"2024-09-05T13:25:57.880019+0000","last_clean":"2024-09-05T13:25:57.880019+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:25:57.880019+0000","last_undegraded":"2024-09-05T13:25:57.880019+0000","last_fullsized":"2024-09-05T13:25:57.880019+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.880166+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:25:57.880166+0000","last_peered":"2024-09-05T13:25:57.880166+0000","last_clean":"2024-09-05T13:25:57.880166+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:25:57.880166+0000","last_undegraded":"2024-09-05T13:25:57.880166+0000","last_fullsized":"2024-09-05T13:25:57.880166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-05T13:25:57.879870+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:25:57.879870+0000","last_peered":"2024-09-05T13:25:57.879870+0000","last_clean":"2024-09-05T13:25:57.879870+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:25:57.879870+0000","last_undegraded":"2024-09-05T13:25:57.879870+0000","last_fullsized":"2024-09-05T13:25:57.879870+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53343,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72568832,"data_stored":72561606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562556,"kb_used_data":71396,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93809284,"statfs":{"total":96636764160,"available":96060706816,"internally_reserved":0,"allocated":73109504,"data_stored":72743242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515368,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856472,"statfs":{"total":96636764160,"available":96109027328,"internally_reserved":0,"allocated":548864,"data_stored":182602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":72564736,"data_stored":72561123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:33.837 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:34.130 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:34.131 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:34.143 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-09-05T13:26:33.788671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53499,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463676,"kb_used_data":70604,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281651844,"statfs":{"total":289910292480,"available":288411488256,"internally_reserved":0,"allocated":72298496,"data_stored":71174565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001512"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29945,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951237+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:26:27.951237+0000","last_peered":"2024-09-05T13:26:27.951237+0000","last_clean":"2024-09-05T13:26:27.951237+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:26:27.951237+0000","last_undegraded":"2024-09-05T13:26:27.951237+0000","last_fullsized":"2024-09-05T13:26:27.951237+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28622,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950403+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:26:27.950403+0000","last_peered":"2024-09-05T13:26:27.950403+0000","last_clean":"2024-09-05T13:26:27.950403+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:26:27.950403+0000","last_undegraded":"2024-09-05T13:26:27.950403+0000","last_fullsized":"2024-09-05T13:26:27.950403+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:26:27.951163+0000","last_peered":"2024-09-05T13:26:27.951163+0000","last_clean":"2024-09-05T13:26:27.951163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:26:27.951163+0000","last_undegraded":"2024-09-05T13:26:27.951163+0000","last_fullsized":"2024-09-05T13:26:27.951163+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.025415+0000","last_change":"2024-09-05T13:26:28.024975+0000","last_active":"2024-09-05T13:26:28.025415+0000","last_peered":"2024-09-05T13:26:28.025415+0000","last_clean":"2024-09-05T13:26:28.025415+0000","last_became_active":"2024-09-05T13:26:27.952269+0000","last_became_peered":"2024-09-05T13:26:27.952269+0000","last_unstale":"2024-09-05T13:26:28.025415+0000","last_undegraded":"2024-09-05T13:26:28.025415+0000","last_fullsized":"2024-09-05T13:26:28.025415+0000","mapping_epoch":66,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:58.257172+0000","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29703,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950133+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:26:27.950133+0000","last_peered":"2024-09-05T13:26:27.950133+0000","last_clean":"2024-09-05T13:26:27.950133+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:26:27.950133+0000","last_undegraded":"2024-09-05T13:26:27.950133+0000","last_fullsized":"2024-09-05T13:26:27.950133+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21284,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950812+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:26:27.950812+0000","last_peered":"2024-09-05T13:26:27.950812+0000","last_clean":"2024-09-05T13:26:27.950812+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:26:27.950812+0000","last_undegraded":"2024-09-05T13:26:27.950812+0000","last_fullsized":"2024-09-05T13:26:27.950812+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950011+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:26:27.950011+0000","last_peered":"2024-09-05T13:26:27.950011+0000","last_clean":"2024-09-05T13:26:27.950011+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:26:27.950011+0000","last_undegraded":"2024-09-05T13:26:27.950011+0000","last_fullsized":"2024-09-05T13:26:27.950011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21275,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950258+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:26:27.950258+0000","last_peered":"2024-09-05T13:26:27.950258+0000","last_clean":"2024-09-05T13:26:27.950258+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:26:27.950258+0000","last_undegraded":"2024-09-05T13:26:27.950258+0000","last_fullsized":"2024-09-05T13:26:27.950258+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30016,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950913+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:26:27.950913+0000","last_peered":"2024-09-05T13:26:27.950913+0000","last_clean":"2024-09-05T13:26:27.950913+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:26:27.950913+0000","last_undegraded":"2024-09-05T13:26:27.950913+0000","last_fullsized":"2024-09-05T13:26:27.950913+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950622+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:26:27.950622+0000","last_peered":"2024-09-05T13:26:27.950622+0000","last_clean":"2024-09-05T13:26:27.950622+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:26:27.950622+0000","last_undegraded":"2024-09-05T13:26:27.950622+0000","last_fullsized":"2024-09-05T13:26:27.950622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950172+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:26:27.950172+0000","last_peered":"2024-09-05T13:26:27.950172+0000","last_clean":"2024-09-05T13:26:27.950172+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:26:27.950172+0000","last_undegraded":"2024-09-05T13:26:27.950172+0000","last_fullsized":"2024-09-05T13:26:27.950172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950871+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:26:27.950871+0000","last_peered":"2024-09-05T13:26:27.950871+0000","last_clean":"2024-09-05T13:26:27.950871+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:26:27.950871+0000","last_undegraded":"2024-09-05T13:26:27.950871+0000","last_fullsized":"2024-09-05T13:26:27.950871+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950397+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:26:27.950397+0000","last_peered":"2024-09-05T13:26:27.950397+0000","last_clean":"2024-09-05T13:26:27.950397+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:26:27.950397+0000","last_undegraded":"2024-09-05T13:26:27.950397+0000","last_fullsized":"2024-09-05T13:26:27.950397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950515+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:26:27.950515+0000","last_peered":"2024-09-05T13:26:27.950515+0000","last_clean":"2024-09-05T13:26:27.950515+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:26:27.950515+0000","last_undegraded":"2024-09-05T13:26:27.950515+0000","last_fullsized":"2024-09-05T13:26:27.950515+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21272,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950412+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:26:27.950412+0000","last_peered":"2024-09-05T13:26:27.950412+0000","last_clean":"2024-09-05T13:26:27.950412+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:26:27.950412+0000","last_undegraded":"2024-09-05T13:26:27.950412+0000","last_fullsized":"2024-09-05T13:26:27.950412+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950776+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:26:27.950776+0000","last_peered":"2024-09-05T13:26:27.950776+0000","last_clean":"2024-09-05T13:26:27.950776+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:26:27.950776+0000","last_undegraded":"2024-09-05T13:26:27.950776+0000","last_fullsized":"2024-09-05T13:26:27.950776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950517+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:26:27.950517+0000","last_peered":"2024-09-05T13:26:27.950517+0000","last_clean":"2024-09-05T13:26:27.950517+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:26:27.950517+0000","last_undegraded":"2024-09-05T13:26:27.950517+0000","last_fullsized":"2024-09-05T13:26:27.950517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21364,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950292+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:26:27.950292+0000","last_peered":"2024-09-05T13:26:27.950292+0000","last_clean":"2024-09-05T13:26:27.950292+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:26:27.950292+0000","last_undegraded":"2024-09-05T13:26:27.950292+0000","last_fullsized":"2024-09-05T13:26:27.950292+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53498,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70127616,"data_stored":70120390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560660,"kb_used_data":69508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93811180,"statfs":{"total":96636764160,"available":96062648320,"internally_reserved":0,"allocated":71176192,"data_stored":70798985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519544,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852296,"statfs":{"total":96636764160,"available":96104751104,"internally_reserved":0,"allocated":565248,"data_stored":188041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70123520,"data_stored":70119907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:34.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-05T13:26:34.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-05T13:26:34.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:34.424 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:34.436 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-09-05T13:26:33.788671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53499,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463676,"kb_used_data":70604,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281651844,"statfs":{"total":289910292480,"available":288411488256,"internally_reserved":0,"allocated":72298496,"data_stored":71174565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001512"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29945,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951237+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:26:27.951237+0000","last_peered":"2024-09-05T13:26:27.951237+0000","last_clean":"2024-09-05T13:26:27.951237+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:26:27.951237+0000","last_undegraded":"2024-09-05T13:26:27.951237+0000","last_fullsized":"2024-09-05T13:26:27.951237+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28622,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950403+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:26:27.950403+0000","last_peered":"2024-09-05T13:26:27.950403+0000","last_clean":"2024-09-05T13:26:27.950403+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:26:27.950403+0000","last_undegraded":"2024-09-05T13:26:27.950403+0000","last_fullsized":"2024-09-05T13:26:27.950403+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:26:27.951163+0000","last_peered":"2024-09-05T13:26:27.951163+0000","last_clean":"2024-09-05T13:26:27.951163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:26:27.951163+0000","last_undegraded":"2024-09-05T13:26:27.951163+0000","last_fullsized":"2024-09-05T13:26:27.951163+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.025415+0000","last_change":"2024-09-05T13:26:28.024975+0000","last_active":"2024-09-05T13:26:28.025415+0000","last_peered":"2024-09-05T13:26:28.025415+0000","last_clean":"2024-09-05T13:26:28.025415+0000","last_became_active":"2024-09-05T13:26:27.952269+0000","last_became_peered":"2024-09-05T13:26:27.952269+0000","last_unstale":"2024-09-05T13:26:28.025415+0000","last_undegraded":"2024-09-05T13:26:28.025415+0000","last_fullsized":"2024-09-05T13:26:28.025415+0000","mapping_epoch":66,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:58.257172+0000","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29703,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950133+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:26:27.950133+0000","last_peered":"2024-09-05T13:26:27.950133+0000","last_clean":"2024-09-05T13:26:27.950133+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:26:27.950133+0000","last_undegraded":"2024-09-05T13:26:27.950133+0000","last_fullsized":"2024-09-05T13:26:27.950133+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21284,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950812+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:26:27.950812+0000","last_peered":"2024-09-05T13:26:27.950812+0000","last_clean":"2024-09-05T13:26:27.950812+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:26:27.950812+0000","last_undegraded":"2024-09-05T13:26:27.950812+0000","last_fullsized":"2024-09-05T13:26:27.950812+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950011+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:26:27.950011+0000","last_peered":"2024-09-05T13:26:27.950011+0000","last_clean":"2024-09-05T13:26:27.950011+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:26:27.950011+0000","last_undegraded":"2024-09-05T13:26:27.950011+0000","last_fullsized":"2024-09-05T13:26:27.950011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21275,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950258+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:26:27.950258+0000","last_peered":"2024-09-05T13:26:27.950258+0000","last_clean":"2024-09-05T13:26:27.950258+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:26:27.950258+0000","last_undegraded":"2024-09-05T13:26:27.950258+0000","last_fullsized":"2024-09-05T13:26:27.950258+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30016,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950913+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:26:27.950913+0000","last_peered":"2024-09-05T13:26:27.950913+0000","last_clean":"2024-09-05T13:26:27.950913+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:26:27.950913+0000","last_undegraded":"2024-09-05T13:26:27.950913+0000","last_fullsized":"2024-09-05T13:26:27.950913+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950622+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:26:27.950622+0000","last_peered":"2024-09-05T13:26:27.950622+0000","last_clean":"2024-09-05T13:26:27.950622+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:26:27.950622+0000","last_undegraded":"2024-09-05T13:26:27.950622+0000","last_fullsized":"2024-09-05T13:26:27.950622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950172+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:26:27.950172+0000","last_peered":"2024-09-05T13:26:27.950172+0000","last_clean":"2024-09-05T13:26:27.950172+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:26:27.950172+0000","last_undegraded":"2024-09-05T13:26:27.950172+0000","last_fullsized":"2024-09-05T13:26:27.950172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950871+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:26:27.950871+0000","last_peered":"2024-09-05T13:26:27.950871+0000","last_clean":"2024-09-05T13:26:27.950871+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:26:27.950871+0000","last_undegraded":"2024-09-05T13:26:27.950871+0000","last_fullsized":"2024-09-05T13:26:27.950871+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950397+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:26:27.950397+0000","last_peered":"2024-09-05T13:26:27.950397+0000","last_clean":"2024-09-05T13:26:27.950397+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:26:27.950397+0000","last_undegraded":"2024-09-05T13:26:27.950397+0000","last_fullsized":"2024-09-05T13:26:27.950397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950515+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:26:27.950515+0000","last_peered":"2024-09-05T13:26:27.950515+0000","last_clean":"2024-09-05T13:26:27.950515+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:26:27.950515+0000","last_undegraded":"2024-09-05T13:26:27.950515+0000","last_fullsized":"2024-09-05T13:26:27.950515+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21272,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950412+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:26:27.950412+0000","last_peered":"2024-09-05T13:26:27.950412+0000","last_clean":"2024-09-05T13:26:27.950412+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:26:27.950412+0000","last_undegraded":"2024-09-05T13:26:27.950412+0000","last_fullsized":"2024-09-05T13:26:27.950412+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950776+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:26:27.950776+0000","last_peered":"2024-09-05T13:26:27.950776+0000","last_clean":"2024-09-05T13:26:27.950776+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:26:27.950776+0000","last_undegraded":"2024-09-05T13:26:27.950776+0000","last_fullsized":"2024-09-05T13:26:27.950776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950517+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:26:27.950517+0000","last_peered":"2024-09-05T13:26:27.950517+0000","last_clean":"2024-09-05T13:26:27.950517+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:26:27.950517+0000","last_undegraded":"2024-09-05T13:26:27.950517+0000","last_fullsized":"2024-09-05T13:26:27.950517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21364,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950292+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:26:27.950292+0000","last_peered":"2024-09-05T13:26:27.950292+0000","last_clean":"2024-09-05T13:26:27.950292+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:26:27.950292+0000","last_undegraded":"2024-09-05T13:26:27.950292+0000","last_fullsized":"2024-09-05T13:26:27.950292+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53498,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70127616,"data_stored":70120390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560660,"kb_used_data":69508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93811180,"statfs":{"total":96636764160,"available":96062648320,"internally_reserved":0,"allocated":71176192,"data_stored":70798985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519544,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852296,"statfs":{"total":96636764160,"available":96104751104,"internally_reserved":0,"allocated":565248,"data_stored":188041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70123520,"data_stored":70119907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:34.437 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:34.717 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:34.718 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:34.730 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-09-05T13:26:33.788671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53499,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463676,"kb_used_data":70604,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281651844,"statfs":{"total":289910292480,"available":288411488256,"internally_reserved":0,"allocated":72298496,"data_stored":71174565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001512"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29945,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951237+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:26:27.951237+0000","last_peered":"2024-09-05T13:26:27.951237+0000","last_clean":"2024-09-05T13:26:27.951237+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:26:27.951237+0000","last_undegraded":"2024-09-05T13:26:27.951237+0000","last_fullsized":"2024-09-05T13:26:27.951237+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28622,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950403+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:26:27.950403+0000","last_peered":"2024-09-05T13:26:27.950403+0000","last_clean":"2024-09-05T13:26:27.950403+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:26:27.950403+0000","last_undegraded":"2024-09-05T13:26:27.950403+0000","last_fullsized":"2024-09-05T13:26:27.950403+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:26:27.951163+0000","last_peered":"2024-09-05T13:26:27.951163+0000","last_clean":"2024-09-05T13:26:27.951163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:26:27.951163+0000","last_undegraded":"2024-09-05T13:26:27.951163+0000","last_fullsized":"2024-09-05T13:26:27.951163+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.025415+0000","last_change":"2024-09-05T13:26:28.024975+0000","last_active":"2024-09-05T13:26:28.025415+0000","last_peered":"2024-09-05T13:26:28.025415+0000","last_clean":"2024-09-05T13:26:28.025415+0000","last_became_active":"2024-09-05T13:26:27.952269+0000","last_became_peered":"2024-09-05T13:26:27.952269+0000","last_unstale":"2024-09-05T13:26:28.025415+0000","last_undegraded":"2024-09-05T13:26:28.025415+0000","last_fullsized":"2024-09-05T13:26:28.025415+0000","mapping_epoch":66,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:58.257172+0000","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29703,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950133+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:26:27.950133+0000","last_peered":"2024-09-05T13:26:27.950133+0000","last_clean":"2024-09-05T13:26:27.950133+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:26:27.950133+0000","last_undegraded":"2024-09-05T13:26:27.950133+0000","last_fullsized":"2024-09-05T13:26:27.950133+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21284,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950812+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:26:27.950812+0000","last_peered":"2024-09-05T13:26:27.950812+0000","last_clean":"2024-09-05T13:26:27.950812+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:26:27.950812+0000","last_undegraded":"2024-09-05T13:26:27.950812+0000","last_fullsized":"2024-09-05T13:26:27.950812+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950011+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:26:27.950011+0000","last_peered":"2024-09-05T13:26:27.950011+0000","last_clean":"2024-09-05T13:26:27.950011+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:26:27.950011+0000","last_undegraded":"2024-09-05T13:26:27.950011+0000","last_fullsized":"2024-09-05T13:26:27.950011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21275,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950258+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:26:27.950258+0000","last_peered":"2024-09-05T13:26:27.950258+0000","last_clean":"2024-09-05T13:26:27.950258+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:26:27.950258+0000","last_undegraded":"2024-09-05T13:26:27.950258+0000","last_fullsized":"2024-09-05T13:26:27.950258+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30016,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950913+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:26:27.950913+0000","last_peered":"2024-09-05T13:26:27.950913+0000","last_clean":"2024-09-05T13:26:27.950913+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:26:27.950913+0000","last_undegraded":"2024-09-05T13:26:27.950913+0000","last_fullsized":"2024-09-05T13:26:27.950913+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950622+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:26:27.950622+0000","last_peered":"2024-09-05T13:26:27.950622+0000","last_clean":"2024-09-05T13:26:27.950622+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:26:27.950622+0000","last_undegraded":"2024-09-05T13:26:27.950622+0000","last_fullsized":"2024-09-05T13:26:27.950622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950172+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:26:27.950172+0000","last_peered":"2024-09-05T13:26:27.950172+0000","last_clean":"2024-09-05T13:26:27.950172+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:26:27.950172+0000","last_undegraded":"2024-09-05T13:26:27.950172+0000","last_fullsized":"2024-09-05T13:26:27.950172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950871+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:26:27.950871+0000","last_peered":"2024-09-05T13:26:27.950871+0000","last_clean":"2024-09-05T13:26:27.950871+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:26:27.950871+0000","last_undegraded":"2024-09-05T13:26:27.950871+0000","last_fullsized":"2024-09-05T13:26:27.950871+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950397+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:26:27.950397+0000","last_peered":"2024-09-05T13:26:27.950397+0000","last_clean":"2024-09-05T13:26:27.950397+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:26:27.950397+0000","last_undegraded":"2024-09-05T13:26:27.950397+0000","last_fullsized":"2024-09-05T13:26:27.950397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950515+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:26:27.950515+0000","last_peered":"2024-09-05T13:26:27.950515+0000","last_clean":"2024-09-05T13:26:27.950515+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:26:27.950515+0000","last_undegraded":"2024-09-05T13:26:27.950515+0000","last_fullsized":"2024-09-05T13:26:27.950515+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21272,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950412+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:26:27.950412+0000","last_peered":"2024-09-05T13:26:27.950412+0000","last_clean":"2024-09-05T13:26:27.950412+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:26:27.950412+0000","last_undegraded":"2024-09-05T13:26:27.950412+0000","last_fullsized":"2024-09-05T13:26:27.950412+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950776+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:26:27.950776+0000","last_peered":"2024-09-05T13:26:27.950776+0000","last_clean":"2024-09-05T13:26:27.950776+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:26:27.950776+0000","last_undegraded":"2024-09-05T13:26:27.950776+0000","last_fullsized":"2024-09-05T13:26:27.950776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950517+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:26:27.950517+0000","last_peered":"2024-09-05T13:26:27.950517+0000","last_clean":"2024-09-05T13:26:27.950517+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:26:27.950517+0000","last_undegraded":"2024-09-05T13:26:27.950517+0000","last_fullsized":"2024-09-05T13:26:27.950517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21364,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950292+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:26:27.950292+0000","last_peered":"2024-09-05T13:26:27.950292+0000","last_clean":"2024-09-05T13:26:27.950292+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:26:27.950292+0000","last_undegraded":"2024-09-05T13:26:27.950292+0000","last_fullsized":"2024-09-05T13:26:27.950292+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53498,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70127616,"data_stored":70120390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560660,"kb_used_data":69508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93811180,"statfs":{"total":96636764160,"available":96062648320,"internally_reserved":0,"allocated":71176192,"data_stored":70798985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519544,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852296,"statfs":{"total":96636764160,"available":96104751104,"internally_reserved":0,"allocated":565248,"data_stored":188041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70123520,"data_stored":70119907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:34.731 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-05T13:26:34.731 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-05T13:26:34.742 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-05T13:26:34.742 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:35.021 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:35.022 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:35.034 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-09-05T13:26:33.788671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53499,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463676,"kb_used_data":70604,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281651844,"statfs":{"total":289910292480,"available":288411488256,"internally_reserved":0,"allocated":72298496,"data_stored":71174565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001512"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29945,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951237+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:26:27.951237+0000","last_peered":"2024-09-05T13:26:27.951237+0000","last_clean":"2024-09-05T13:26:27.951237+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:26:27.951237+0000","last_undegraded":"2024-09-05T13:26:27.951237+0000","last_fullsized":"2024-09-05T13:26:27.951237+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28622,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950403+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:26:27.950403+0000","last_peered":"2024-09-05T13:26:27.950403+0000","last_clean":"2024-09-05T13:26:27.950403+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:26:27.950403+0000","last_undegraded":"2024-09-05T13:26:27.950403+0000","last_fullsized":"2024-09-05T13:26:27.950403+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:26:27.951163+0000","last_peered":"2024-09-05T13:26:27.951163+0000","last_clean":"2024-09-05T13:26:27.951163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:26:27.951163+0000","last_undegraded":"2024-09-05T13:26:27.951163+0000","last_fullsized":"2024-09-05T13:26:27.951163+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.025415+0000","last_change":"2024-09-05T13:26:28.024975+0000","last_active":"2024-09-05T13:26:28.025415+0000","last_peered":"2024-09-05T13:26:28.025415+0000","last_clean":"2024-09-05T13:26:28.025415+0000","last_became_active":"2024-09-05T13:26:27.952269+0000","last_became_peered":"2024-09-05T13:26:27.952269+0000","last_unstale":"2024-09-05T13:26:28.025415+0000","last_undegraded":"2024-09-05T13:26:28.025415+0000","last_fullsized":"2024-09-05T13:26:28.025415+0000","mapping_epoch":66,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:58.257172+0000","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29703,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950133+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:26:27.950133+0000","last_peered":"2024-09-05T13:26:27.950133+0000","last_clean":"2024-09-05T13:26:27.950133+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:26:27.950133+0000","last_undegraded":"2024-09-05T13:26:27.950133+0000","last_fullsized":"2024-09-05T13:26:27.950133+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21284,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950812+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:26:27.950812+0000","last_peered":"2024-09-05T13:26:27.950812+0000","last_clean":"2024-09-05T13:26:27.950812+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:26:27.950812+0000","last_undegraded":"2024-09-05T13:26:27.950812+0000","last_fullsized":"2024-09-05T13:26:27.950812+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950011+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:26:27.950011+0000","last_peered":"2024-09-05T13:26:27.950011+0000","last_clean":"2024-09-05T13:26:27.950011+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:26:27.950011+0000","last_undegraded":"2024-09-05T13:26:27.950011+0000","last_fullsized":"2024-09-05T13:26:27.950011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21275,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950258+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:26:27.950258+0000","last_peered":"2024-09-05T13:26:27.950258+0000","last_clean":"2024-09-05T13:26:27.950258+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:26:27.950258+0000","last_undegraded":"2024-09-05T13:26:27.950258+0000","last_fullsized":"2024-09-05T13:26:27.950258+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30016,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950913+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:26:27.950913+0000","last_peered":"2024-09-05T13:26:27.950913+0000","last_clean":"2024-09-05T13:26:27.950913+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:26:27.950913+0000","last_undegraded":"2024-09-05T13:26:27.950913+0000","last_fullsized":"2024-09-05T13:26:27.950913+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950622+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:26:27.950622+0000","last_peered":"2024-09-05T13:26:27.950622+0000","last_clean":"2024-09-05T13:26:27.950622+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:26:27.950622+0000","last_undegraded":"2024-09-05T13:26:27.950622+0000","last_fullsized":"2024-09-05T13:26:27.950622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950172+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:26:27.950172+0000","last_peered":"2024-09-05T13:26:27.950172+0000","last_clean":"2024-09-05T13:26:27.950172+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:26:27.950172+0000","last_undegraded":"2024-09-05T13:26:27.950172+0000","last_fullsized":"2024-09-05T13:26:27.950172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950871+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:26:27.950871+0000","last_peered":"2024-09-05T13:26:27.950871+0000","last_clean":"2024-09-05T13:26:27.950871+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:26:27.950871+0000","last_undegraded":"2024-09-05T13:26:27.950871+0000","last_fullsized":"2024-09-05T13:26:27.950871+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950397+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:26:27.950397+0000","last_peered":"2024-09-05T13:26:27.950397+0000","last_clean":"2024-09-05T13:26:27.950397+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:26:27.950397+0000","last_undegraded":"2024-09-05T13:26:27.950397+0000","last_fullsized":"2024-09-05T13:26:27.950397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950515+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:26:27.950515+0000","last_peered":"2024-09-05T13:26:27.950515+0000","last_clean":"2024-09-05T13:26:27.950515+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:26:27.950515+0000","last_undegraded":"2024-09-05T13:26:27.950515+0000","last_fullsized":"2024-09-05T13:26:27.950515+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21272,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950412+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:26:27.950412+0000","last_peered":"2024-09-05T13:26:27.950412+0000","last_clean":"2024-09-05T13:26:27.950412+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:26:27.950412+0000","last_undegraded":"2024-09-05T13:26:27.950412+0000","last_fullsized":"2024-09-05T13:26:27.950412+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950776+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:26:27.950776+0000","last_peered":"2024-09-05T13:26:27.950776+0000","last_clean":"2024-09-05T13:26:27.950776+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:26:27.950776+0000","last_undegraded":"2024-09-05T13:26:27.950776+0000","last_fullsized":"2024-09-05T13:26:27.950776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950517+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:26:27.950517+0000","last_peered":"2024-09-05T13:26:27.950517+0000","last_clean":"2024-09-05T13:26:27.950517+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:26:27.950517+0000","last_undegraded":"2024-09-05T13:26:27.950517+0000","last_fullsized":"2024-09-05T13:26:27.950517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21364,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950292+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:26:27.950292+0000","last_peered":"2024-09-05T13:26:27.950292+0000","last_clean":"2024-09-05T13:26:27.950292+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:26:27.950292+0000","last_undegraded":"2024-09-05T13:26:27.950292+0000","last_fullsized":"2024-09-05T13:26:27.950292+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53498,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70127616,"data_stored":70120390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560660,"kb_used_data":69508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93811180,"statfs":{"total":96636764160,"available":96062648320,"internally_reserved":0,"allocated":71176192,"data_stored":70798985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519544,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852296,"statfs":{"total":96636764160,"available":96104751104,"internally_reserved":0,"allocated":565248,"data_stored":188041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70123520,"data_stored":70119907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:35.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:35.313 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:35.313 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:35.324 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-09-05T13:26:33.788671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53499,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463676,"kb_used_data":70604,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281651844,"statfs":{"total":289910292480,"available":288411488256,"internally_reserved":0,"allocated":72298496,"data_stored":71174565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001512"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29945,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951237+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:26:27.951237+0000","last_peered":"2024-09-05T13:26:27.951237+0000","last_clean":"2024-09-05T13:26:27.951237+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:26:27.951237+0000","last_undegraded":"2024-09-05T13:26:27.951237+0000","last_fullsized":"2024-09-05T13:26:27.951237+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28622,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950403+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:26:27.950403+0000","last_peered":"2024-09-05T13:26:27.950403+0000","last_clean":"2024-09-05T13:26:27.950403+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:26:27.950403+0000","last_undegraded":"2024-09-05T13:26:27.950403+0000","last_fullsized":"2024-09-05T13:26:27.950403+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:26:27.951163+0000","last_peered":"2024-09-05T13:26:27.951163+0000","last_clean":"2024-09-05T13:26:27.951163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:26:27.951163+0000","last_undegraded":"2024-09-05T13:26:27.951163+0000","last_fullsized":"2024-09-05T13:26:27.951163+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.025415+0000","last_change":"2024-09-05T13:26:28.024975+0000","last_active":"2024-09-05T13:26:28.025415+0000","last_peered":"2024-09-05T13:26:28.025415+0000","last_clean":"2024-09-05T13:26:28.025415+0000","last_became_active":"2024-09-05T13:26:27.952269+0000","last_became_peered":"2024-09-05T13:26:27.952269+0000","last_unstale":"2024-09-05T13:26:28.025415+0000","last_undegraded":"2024-09-05T13:26:28.025415+0000","last_fullsized":"2024-09-05T13:26:28.025415+0000","mapping_epoch":66,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:58.257172+0000","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29703,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950133+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:26:27.950133+0000","last_peered":"2024-09-05T13:26:27.950133+0000","last_clean":"2024-09-05T13:26:27.950133+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:26:27.950133+0000","last_undegraded":"2024-09-05T13:26:27.950133+0000","last_fullsized":"2024-09-05T13:26:27.950133+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21284,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950812+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:26:27.950812+0000","last_peered":"2024-09-05T13:26:27.950812+0000","last_clean":"2024-09-05T13:26:27.950812+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:26:27.950812+0000","last_undegraded":"2024-09-05T13:26:27.950812+0000","last_fullsized":"2024-09-05T13:26:27.950812+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950011+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:26:27.950011+0000","last_peered":"2024-09-05T13:26:27.950011+0000","last_clean":"2024-09-05T13:26:27.950011+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:26:27.950011+0000","last_undegraded":"2024-09-05T13:26:27.950011+0000","last_fullsized":"2024-09-05T13:26:27.950011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21275,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950258+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:26:27.950258+0000","last_peered":"2024-09-05T13:26:27.950258+0000","last_clean":"2024-09-05T13:26:27.950258+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:26:27.950258+0000","last_undegraded":"2024-09-05T13:26:27.950258+0000","last_fullsized":"2024-09-05T13:26:27.950258+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30016,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950913+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:26:27.950913+0000","last_peered":"2024-09-05T13:26:27.950913+0000","last_clean":"2024-09-05T13:26:27.950913+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:26:27.950913+0000","last_undegraded":"2024-09-05T13:26:27.950913+0000","last_fullsized":"2024-09-05T13:26:27.950913+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950622+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:26:27.950622+0000","last_peered":"2024-09-05T13:26:27.950622+0000","last_clean":"2024-09-05T13:26:27.950622+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:26:27.950622+0000","last_undegraded":"2024-09-05T13:26:27.950622+0000","last_fullsized":"2024-09-05T13:26:27.950622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950172+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:26:27.950172+0000","last_peered":"2024-09-05T13:26:27.950172+0000","last_clean":"2024-09-05T13:26:27.950172+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:26:27.950172+0000","last_undegraded":"2024-09-05T13:26:27.950172+0000","last_fullsized":"2024-09-05T13:26:27.950172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950871+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:26:27.950871+0000","last_peered":"2024-09-05T13:26:27.950871+0000","last_clean":"2024-09-05T13:26:27.950871+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:26:27.950871+0000","last_undegraded":"2024-09-05T13:26:27.950871+0000","last_fullsized":"2024-09-05T13:26:27.950871+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950397+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:26:27.950397+0000","last_peered":"2024-09-05T13:26:27.950397+0000","last_clean":"2024-09-05T13:26:27.950397+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:26:27.950397+0000","last_undegraded":"2024-09-05T13:26:27.950397+0000","last_fullsized":"2024-09-05T13:26:27.950397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950515+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:26:27.950515+0000","last_peered":"2024-09-05T13:26:27.950515+0000","last_clean":"2024-09-05T13:26:27.950515+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:26:27.950515+0000","last_undegraded":"2024-09-05T13:26:27.950515+0000","last_fullsized":"2024-09-05T13:26:27.950515+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21272,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950412+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:26:27.950412+0000","last_peered":"2024-09-05T13:26:27.950412+0000","last_clean":"2024-09-05T13:26:27.950412+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:26:27.950412+0000","last_undegraded":"2024-09-05T13:26:27.950412+0000","last_fullsized":"2024-09-05T13:26:27.950412+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950776+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:26:27.950776+0000","last_peered":"2024-09-05T13:26:27.950776+0000","last_clean":"2024-09-05T13:26:27.950776+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:26:27.950776+0000","last_undegraded":"2024-09-05T13:26:27.950776+0000","last_fullsized":"2024-09-05T13:26:27.950776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950517+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:26:27.950517+0000","last_peered":"2024-09-05T13:26:27.950517+0000","last_clean":"2024-09-05T13:26:27.950517+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:26:27.950517+0000","last_undegraded":"2024-09-05T13:26:27.950517+0000","last_fullsized":"2024-09-05T13:26:27.950517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21364,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950292+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:26:27.950292+0000","last_peered":"2024-09-05T13:26:27.950292+0000","last_clean":"2024-09-05T13:26:27.950292+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:26:27.950292+0000","last_undegraded":"2024-09-05T13:26:27.950292+0000","last_fullsized":"2024-09-05T13:26:27.950292+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53498,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70127616,"data_stored":70120390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560660,"kb_used_data":69508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93811180,"statfs":{"total":96636764160,"available":96062648320,"internally_reserved":0,"allocated":71176192,"data_stored":70798985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519544,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852296,"statfs":{"total":96636764160,"available":96104751104,"internally_reserved":0,"allocated":565248,"data_stored":188041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70123520,"data_stored":70119907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:35.326 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-05T13:26:35.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:35.608 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:35.609 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:35.621 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-09-05T13:26:33.788671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53499,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463676,"kb_used_data":70604,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281651844,"statfs":{"total":289910292480,"available":288411488256,"internally_reserved":0,"allocated":72298496,"data_stored":71174565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001512"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29945,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951237+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:26:27.951237+0000","last_peered":"2024-09-05T13:26:27.951237+0000","last_clean":"2024-09-05T13:26:27.951237+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:26:27.951237+0000","last_undegraded":"2024-09-05T13:26:27.951237+0000","last_fullsized":"2024-09-05T13:26:27.951237+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28622,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950403+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:26:27.950403+0000","last_peered":"2024-09-05T13:26:27.950403+0000","last_clean":"2024-09-05T13:26:27.950403+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:26:27.950403+0000","last_undegraded":"2024-09-05T13:26:27.950403+0000","last_fullsized":"2024-09-05T13:26:27.950403+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:26:27.951163+0000","last_peered":"2024-09-05T13:26:27.951163+0000","last_clean":"2024-09-05T13:26:27.951163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:26:27.951163+0000","last_undegraded":"2024-09-05T13:26:27.951163+0000","last_fullsized":"2024-09-05T13:26:27.951163+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.025415+0000","last_change":"2024-09-05T13:26:28.024975+0000","last_active":"2024-09-05T13:26:28.025415+0000","last_peered":"2024-09-05T13:26:28.025415+0000","last_clean":"2024-09-05T13:26:28.025415+0000","last_became_active":"2024-09-05T13:26:27.952269+0000","last_became_peered":"2024-09-05T13:26:27.952269+0000","last_unstale":"2024-09-05T13:26:28.025415+0000","last_undegraded":"2024-09-05T13:26:28.025415+0000","last_fullsized":"2024-09-05T13:26:28.025415+0000","mapping_epoch":66,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:58.257172+0000","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29703,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950133+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:26:27.950133+0000","last_peered":"2024-09-05T13:26:27.950133+0000","last_clean":"2024-09-05T13:26:27.950133+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:26:27.950133+0000","last_undegraded":"2024-09-05T13:26:27.950133+0000","last_fullsized":"2024-09-05T13:26:27.950133+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21284,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950812+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:26:27.950812+0000","last_peered":"2024-09-05T13:26:27.950812+0000","last_clean":"2024-09-05T13:26:27.950812+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:26:27.950812+0000","last_undegraded":"2024-09-05T13:26:27.950812+0000","last_fullsized":"2024-09-05T13:26:27.950812+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950011+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:26:27.950011+0000","last_peered":"2024-09-05T13:26:27.950011+0000","last_clean":"2024-09-05T13:26:27.950011+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:26:27.950011+0000","last_undegraded":"2024-09-05T13:26:27.950011+0000","last_fullsized":"2024-09-05T13:26:27.950011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21275,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950258+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:26:27.950258+0000","last_peered":"2024-09-05T13:26:27.950258+0000","last_clean":"2024-09-05T13:26:27.950258+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:26:27.950258+0000","last_undegraded":"2024-09-05T13:26:27.950258+0000","last_fullsized":"2024-09-05T13:26:27.950258+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30016,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950913+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:26:27.950913+0000","last_peered":"2024-09-05T13:26:27.950913+0000","last_clean":"2024-09-05T13:26:27.950913+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:26:27.950913+0000","last_undegraded":"2024-09-05T13:26:27.950913+0000","last_fullsized":"2024-09-05T13:26:27.950913+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950622+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:26:27.950622+0000","last_peered":"2024-09-05T13:26:27.950622+0000","last_clean":"2024-09-05T13:26:27.950622+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:26:27.950622+0000","last_undegraded":"2024-09-05T13:26:27.950622+0000","last_fullsized":"2024-09-05T13:26:27.950622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950172+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:26:27.950172+0000","last_peered":"2024-09-05T13:26:27.950172+0000","last_clean":"2024-09-05T13:26:27.950172+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:26:27.950172+0000","last_undegraded":"2024-09-05T13:26:27.950172+0000","last_fullsized":"2024-09-05T13:26:27.950172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950871+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:26:27.950871+0000","last_peered":"2024-09-05T13:26:27.950871+0000","last_clean":"2024-09-05T13:26:27.950871+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:26:27.950871+0000","last_undegraded":"2024-09-05T13:26:27.950871+0000","last_fullsized":"2024-09-05T13:26:27.950871+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950397+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:26:27.950397+0000","last_peered":"2024-09-05T13:26:27.950397+0000","last_clean":"2024-09-05T13:26:27.950397+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:26:27.950397+0000","last_undegraded":"2024-09-05T13:26:27.950397+0000","last_fullsized":"2024-09-05T13:26:27.950397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950515+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:26:27.950515+0000","last_peered":"2024-09-05T13:26:27.950515+0000","last_clean":"2024-09-05T13:26:27.950515+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:26:27.950515+0000","last_undegraded":"2024-09-05T13:26:27.950515+0000","last_fullsized":"2024-09-05T13:26:27.950515+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21272,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950412+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:26:27.950412+0000","last_peered":"2024-09-05T13:26:27.950412+0000","last_clean":"2024-09-05T13:26:27.950412+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:26:27.950412+0000","last_undegraded":"2024-09-05T13:26:27.950412+0000","last_fullsized":"2024-09-05T13:26:27.950412+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950776+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:26:27.950776+0000","last_peered":"2024-09-05T13:26:27.950776+0000","last_clean":"2024-09-05T13:26:27.950776+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:26:27.950776+0000","last_undegraded":"2024-09-05T13:26:27.950776+0000","last_fullsized":"2024-09-05T13:26:27.950776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950517+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:26:27.950517+0000","last_peered":"2024-09-05T13:26:27.950517+0000","last_clean":"2024-09-05T13:26:27.950517+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:26:27.950517+0000","last_undegraded":"2024-09-05T13:26:27.950517+0000","last_fullsized":"2024-09-05T13:26:27.950517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21364,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950292+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:26:27.950292+0000","last_peered":"2024-09-05T13:26:27.950292+0000","last_clean":"2024-09-05T13:26:27.950292+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:26:27.950292+0000","last_undegraded":"2024-09-05T13:26:27.950292+0000","last_fullsized":"2024-09-05T13:26:27.950292+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53498,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70127616,"data_stored":70120390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":560660,"kb_used_data":69508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93811180,"statfs":{"total":96636764160,"available":96062648320,"internally_reserved":0,"allocated":71176192,"data_stored":70798985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519544,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852296,"statfs":{"total":96636764160,"available":96104751104,"internally_reserved":0,"allocated":565248,"data_stored":188041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70123520,"data_stored":70119907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:35.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T13:26:35.908 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:35.908 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"fsid":"0162b639-bdbb-4bd7-a784-515f78b22316","created":"2024-09-05T13:11:13.133363+0000","modified":"2024-09-05T13:26:27.942172+0000","last_up_change":"2024-09-05T13:13:15.880253+0000","last_in_change":"2024-09-05T13:13:07.720441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-05T13:11:28.452062+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-05T13:11:39.270087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df7aef1c-c387-49d7-b4b7-627cbd8ab67e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":65,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6804","nonce":2531722421}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6805","nonce":2531722421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6807","nonce":2531722421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6806","nonce":2531722421}]},"public_addr":"172.21.15.86:6804/2531722421","cluster_addr":"172.21.15.86:6805/2531722421","heartbeat_back_addr":"172.21.15.86:6807/2531722421","heartbeat_front_addr":"172.21.15.86:6806/2531722421","state":["exists","up"]},{"osd":1,"uuid":"623d69d0-4fe7-431e-aa67-d7239d6aef62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":26,"up_from":41,"up_thru":66,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6808","nonce":4255212274}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6809","nonce":4255212274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6811","nonce":4255212274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6810","nonce":4255212274}]},"public_addr":"172.21.15.86:6808/4255212274","cluster_addr":"172.21.15.86:6809/4255212274","heartbeat_back_addr":"172.21.15.86:6811/4255212274","heartbeat_front_addr":"172.21.15.86:6810/4255212274","state":["exists","up"]},{"osd":2,"uuid":"b7c8e230-5ce4-4513-b07a-64cfea1afc1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":17,"up_from":31,"up_thru":56,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6800","nonce":2895960470}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6801","nonce":2895960470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6803","nonce":2895960470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.86:6802","nonce":2895960470}]},"public_addr":"172.21.15.86:6800/2895960470","cluster_addr":"172.21.15.86:6801/2895960470","heartbeat_back_addr":"172.21.15.86:6803/2895960470","heartbeat_front_addr":"172.21.15.86:6802/2895960470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.643488+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:20.440515+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T13:11:21.020419+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T13:26:36.921 INFO:tasks.ceph:Scrubbing osd.0 2024-09-05T13:26:36.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-05T13:26:37.048 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:26:37.048 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-05T13:26:37.048 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:26:37.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-05T13:26:37.340 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 0 to deep-scrub 2024-09-05T13:26:37.353 INFO:tasks.ceph:Scrubbing osd.1 2024-09-05T13:26:37.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-05T13:26:37.474 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:26:37.474 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-05T13:26:37.474 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:26:37.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-05T13:26:37.766 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 1 to deep-scrub 2024-09-05T13:26:37.779 INFO:tasks.ceph:Scrubbing osd.2 2024-09-05T13:26:37.780 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-05T13:26:37.901 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-09-05T13:26:37.901 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-05T13:26:37.901 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-09-05T13:26:37.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-05T13:26:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 2 to deep-scrub 2024-09-05T13:26:38.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:38.487 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:38.488 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:38.499 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-09-05T13:26:37.789298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53499,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461276,"kb_used_data":68204,"kb_used_omap":12,"kb_used_meta":1393011,"kb_avail":281654244,"statfs":{"total":289910292480,"available":288413945856,"internally_reserved":0,"allocated":69840896,"data_stored":68716965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1426443270},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001722"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29945,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951237+0000","last_change":"2024-09-05T13:24:03.329380+0000","last_active":"2024-09-05T13:26:27.951237+0000","last_peered":"2024-09-05T13:26:27.951237+0000","last_clean":"2024-09-05T13:26:27.951237+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:26:27.951237+0000","last_undegraded":"2024-09-05T13:26:27.951237+0000","last_fullsized":"2024-09-05T13:26:27.951237+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:36:56.923427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20907,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067987+0000","last_change":"2024-09-05T13:12:46.050207+0000","last_active":"2024-09-05T13:26:28.067987+0000","last_peered":"2024-09-05T13:26:28.067987+0000","last_clean":"2024-09-05T13:26:28.067987+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:28.067987+0000","last_undegraded":"2024-09-05T13:26:28.067987+0000","last_fullsized":"2024-09-05T13:26:28.067987+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2031","last_scrub_stamp":"2024-09-05T13:12:46.049765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:46.049765+0000","objects_scrubbed":1984,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:26.639249+0000","scrub_duration":794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28622,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950403+0000","last_change":"2024-09-05T13:22:50.969316+0000","last_active":"2024-09-05T13:26:27.950403+0000","last_peered":"2024-09-05T13:26:27.950403+0000","last_clean":"2024-09-05T13:26:27.950403+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:26:27.950403+0000","last_undegraded":"2024-09-05T13:26:27.950403+0000","last_fullsized":"2024-09-05T13:26:27.950403+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2236","last_scrub_stamp":"2024-09-05T13:12:47.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:47.044046+0000","objects_scrubbed":2185,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:58:21.685298+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067957+0000","last_change":"2024-09-05T13:12:25.167256+0000","last_active":"2024-09-05T13:26:28.067957+0000","last_peered":"2024-09-05T13:26:28.067957+0000","last_clean":"2024-09-05T13:26:28.067957+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:28.067957+0000","last_undegraded":"2024-09-05T13:26:28.067957+0000","last_fullsized":"2024-09-05T13:26:28.067957+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:11:39.923202+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:09.468396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21066,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.951163+0000","last_change":"2024-09-05T13:13:01.524689+0000","last_active":"2024-09-05T13:26:27.951163+0000","last_peered":"2024-09-05T13:26:27.951163+0000","last_clean":"2024-09-05T13:26:27.951163+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:26:27.951163+0000","last_undegraded":"2024-09-05T13:26:27.951163+0000","last_fullsized":"2024-09-05T13:26:27.951163+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3523","last_scrub_stamp":"2024-09-05T13:13:01.524564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:01.524564+0000","objects_scrubbed":3453,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:02.536348+0000","scrub_duration":1438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21310,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.025415+0000","last_change":"2024-09-05T13:26:28.024975+0000","last_active":"2024-09-05T13:26:28.025415+0000","last_peered":"2024-09-05T13:26:28.025415+0000","last_clean":"2024-09-05T13:26:28.025415+0000","last_became_active":"2024-09-05T13:26:27.952269+0000","last_became_peered":"2024-09-05T13:26:27.952269+0000","last_unstale":"2024-09-05T13:26:28.025415+0000","last_undegraded":"2024-09-05T13:26:28.025415+0000","last_fullsized":"2024-09-05T13:26:28.025415+0000","mapping_epoch":66,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3639","last_scrub_stamp":"2024-09-05T13:13:02.937876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:02.937876+0000","objects_scrubbed":3540,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:58.257172+0000","scrub_duration":1813,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29703,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950133+0000","last_change":"2024-09-05T13:19:36.151660+0000","last_active":"2024-09-05T13:26:27.950133+0000","last_peered":"2024-09-05T13:26:27.950133+0000","last_clean":"2024-09-05T13:26:27.950133+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:26:27.950133+0000","last_undegraded":"2024-09-05T13:26:27.950133+0000","last_fullsized":"2024-09-05T13:26:27.950133+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3737","last_scrub_stamp":"2024-09-05T13:13:04.539372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.539372+0000","objects_scrubbed":3675,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:12.605585+0000","scrub_duration":1443,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21284,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950812+0000","last_change":"2024-09-05T13:13:00.435185+0000","last_active":"2024-09-05T13:26:27.950812+0000","last_peered":"2024-09-05T13:26:27.950812+0000","last_clean":"2024-09-05T13:26:27.950812+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:26:27.950812+0000","last_undegraded":"2024-09-05T13:26:27.950812+0000","last_fullsized":"2024-09-05T13:26:27.950812+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-05T13:13:00.434573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:00.434573+0000","objects_scrubbed":3388,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:43.000995+0000","scrub_duration":1396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950011+0000","last_change":"2024-09-05T13:13:17.912312+0000","last_active":"2024-09-05T13:26:27.950011+0000","last_peered":"2024-09-05T13:26:27.950011+0000","last_clean":"2024-09-05T13:26:27.950011+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:26:27.950011+0000","last_undegraded":"2024-09-05T13:26:27.950011+0000","last_fullsized":"2024-09-05T13:26:27.950011+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:42:47.804322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21275,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950258+0000","last_change":"2024-09-05T13:23:41.325577+0000","last_active":"2024-09-05T13:26:27.950258+0000","last_peered":"2024-09-05T13:26:27.950258+0000","last_clean":"2024-09-05T13:26:27.950258+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:26:27.950258+0000","last_undegraded":"2024-09-05T13:26:27.950258+0000","last_fullsized":"2024-09-05T13:26:27.950258+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2383","last_scrub_stamp":"2024-09-05T13:12:49.049957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:49.049957+0000","objects_scrubbed":2339,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:58:42.163591+0000","scrub_duration":916,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067857+0000","last_change":"2024-09-05T13:12:25.165250+0000","last_active":"2024-09-05T13:26:28.067857+0000","last_peered":"2024-09-05T13:26:28.067857+0000","last_clean":"2024-09-05T13:26:28.067857+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:28.067857+0000","last_undegraded":"2024-09-05T13:26:28.067857+0000","last_fullsized":"2024-09-05T13:26:28.067857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:13.415404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30016,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950913+0000","last_change":"2024-09-05T13:24:49.438075+0000","last_active":"2024-09-05T13:26:27.950913+0000","last_peered":"2024-09-05T13:26:27.950913+0000","last_clean":"2024-09-05T13:26:27.950913+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:26:27.950913+0000","last_undegraded":"2024-09-05T13:26:27.950913+0000","last_fullsized":"2024-09-05T13:26:27.950913+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3709","last_scrub_stamp":"2024-09-05T13:13:04.159129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:04.159129+0000","objects_scrubbed":3625,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:55:06.933831+0000","scrub_duration":2037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950622+0000","last_change":"2024-09-05T13:13:18.017066+0000","last_active":"2024-09-05T13:26:27.950622+0000","last_peered":"2024-09-05T13:26:27.950622+0000","last_clean":"2024-09-05T13:26:27.950622+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:26:27.950622+0000","last_undegraded":"2024-09-05T13:26:27.950622+0000","last_fullsized":"2024-09-05T13:26:27.950622+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:04.574521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29860,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067911+0000","last_change":"2024-09-05T13:16:09.214551+0000","last_active":"2024-09-05T13:26:28.067911+0000","last_peered":"2024-09-05T13:26:28.067911+0000","last_clean":"2024-09-05T13:26:28.067911+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:28.067911+0000","last_undegraded":"2024-09-05T13:26:28.067911+0000","last_fullsized":"2024-09-05T13:26:28.067911+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2441","last_scrub_stamp":"2024-09-05T13:12:50.064199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:50.064199+0000","objects_scrubbed":2400,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:58:09.759191+0000","scrub_duration":873,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950172+0000","last_change":"2024-09-05T13:13:17.975563+0000","last_active":"2024-09-05T13:26:27.950172+0000","last_peered":"2024-09-05T13:26:27.950172+0000","last_clean":"2024-09-05T13:26:27.950172+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:26:27.950172+0000","last_undegraded":"2024-09-05T13:26:27.950172+0000","last_fullsized":"2024-09-05T13:26:27.950172+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:19:37.976017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29540,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067848+0000","last_change":"2024-09-05T13:24:22.397213+0000","last_active":"2024-09-05T13:26:28.067848+0000","last_peered":"2024-09-05T13:26:28.067848+0000","last_clean":"2024-09-05T13:26:28.067848+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:28.067848+0000","last_undegraded":"2024-09-05T13:26:28.067848+0000","last_fullsized":"2024-09-05T13:26:28.067848+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2537","last_scrub_stamp":"2024-09-05T13:12:51.179242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:51.179242+0000","objects_scrubbed":2485,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:28:35.672803+0000","scrub_duration":1019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21035,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950871+0000","last_change":"2024-09-05T13:25:57.964666+0000","last_active":"2024-09-05T13:26:27.950871+0000","last_peered":"2024-09-05T13:26:27.950871+0000","last_clean":"2024-09-05T13:26:27.950871+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:26:27.950871+0000","last_undegraded":"2024-09-05T13:26:27.950871+0000","last_fullsized":"2024-09-05T13:26:27.950871+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3905","last_scrub_stamp":"2024-09-05T13:13:06.992813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:06.992813+0000","objects_scrubbed":3847,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:31:50.943631+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950397+0000","last_change":"2024-09-05T13:13:18.105555+0000","last_active":"2024-09-05T13:26:27.950397+0000","last_peered":"2024-09-05T13:26:27.950397+0000","last_clean":"2024-09-05T13:26:27.950397+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:26:27.950397+0000","last_undegraded":"2024-09-05T13:26:27.950397+0000","last_fullsized":"2024-09-05T13:26:27.950397+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:46.688760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21319,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950515+0000","last_change":"2024-09-05T13:25:28.584604+0000","last_active":"2024-09-05T13:26:27.950515+0000","last_peered":"2024-09-05T13:26:27.950515+0000","last_clean":"2024-09-05T13:26:27.950515+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:26:27.950515+0000","last_undegraded":"2024-09-05T13:26:27.950515+0000","last_fullsized":"2024-09-05T13:26:27.950515+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'4156","last_scrub_stamp":"2024-09-05T13:13:11.658445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:11.658445+0000","objects_scrubbed":4093,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:39.398293+0000","scrub_duration":1558,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:28.067999+0000","last_change":"2024-09-05T13:12:25.166648+0000","last_active":"2024-09-05T13:26:28.067999+0000","last_peered":"2024-09-05T13:26:28.067999+0000","last_clean":"2024-09-05T13:26:28.067999+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:28.067999+0000","last_undegraded":"2024-09-05T13:26:28.067999+0000","last_fullsized":"2024-09-05T13:26:28.067999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:57:10.078182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21272,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950412+0000","last_change":"2024-09-05T13:23:22.189079+0000","last_active":"2024-09-05T13:26:27.950412+0000","last_peered":"2024-09-05T13:26:27.950412+0000","last_clean":"2024-09-05T13:26:27.950412+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:26:27.950412+0000","last_undegraded":"2024-09-05T13:26:27.950412+0000","last_fullsized":"2024-09-05T13:26:27.950412+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3904","last_scrub_stamp":"2024-09-05T13:13:05.462886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:13:05.462886+0000","objects_scrubbed":3866,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:42:24.355683+0000","scrub_duration":1394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950776+0000","last_change":"2024-09-05T13:13:18.036134+0000","last_active":"2024-09-05T13:26:27.950776+0000","last_peered":"2024-09-05T13:26:27.950776+0000","last_clean":"2024-09-05T13:26:27.950776+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:26:27.950776+0000","last_undegraded":"2024-09-05T13:26:27.950776+0000","last_fullsized":"2024-09-05T13:26:27.950776+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:02:57.473794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950517+0000","last_change":"2024-09-05T13:13:17.906078+0000","last_active":"2024-09-05T13:26:27.950517+0000","last_peered":"2024-09-05T13:26:27.950517+0000","last_clean":"2024-09-05T13:26:27.950517+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:26:27.950517+0000","last_undegraded":"2024-09-05T13:26:27.950517+0000","last_fullsized":"2024-09-05T13:26:27.950517+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:28.526981+0000","last_clean_scrub_stamp":"2024-09-05T13:11:28.526981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:45:26.842320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'21192","reported_seq":21364,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:27.950292+0000","last_change":"2024-09-05T13:25:10.551880+0000","last_active":"2024-09-05T13:26:27.950292+0000","last_peered":"2024-09-05T13:26:27.950292+0000","last_clean":"2024-09-05T13:26:27.950292+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:26:27.950292+0000","last_undegraded":"2024-09-05T13:26:27.950292+0000","last_fullsized":"2024-09-05T13:26:27.950292+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3398","last_scrub_stamp":"2024-09-05T13:12:59.497416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:11:39.923202+0000","last_clean_scrub_stamp":"2024-09-05T13:12:59.497416+0000","objects_scrubbed":3321,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:14:26.288713+0000","scrub_duration":1460,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53498,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":67670016,"data_stored":67662790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558260,"kb_used_data":67108,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93813580,"statfs":{"total":96636764160,"available":96065105920,"internally_reserved":0,"allocated":68718592,"data_stored":68341385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":502918826},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519544,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852296,"statfs":{"total":96636764160,"available":96104751104,"internally_reserved":0,"allocated":565248,"data_stored":188041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67665920,"data_stored":67662307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:38.504 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-05T13:11:39.923202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=39, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.504 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-05T13:12:46.049765+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=12, tm_sec=46, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.504 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-05T13:12:47.044046+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=12, tm_sec=47, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.504 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-05T13:11:39.923202+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=39, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.504 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-05T13:13:01.524564+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=13, tm_sec=1, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.504 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-05T13:13:02.937876+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=13, tm_sec=2, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.504 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-05T13:13:04.539372+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=13, tm_sec=4, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-05T13:13:00.434573+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=13, tm_sec=0, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-05T13:11:28.526981+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=28, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-05T13:12:49.049957+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=12, tm_sec=49, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-05T13:11:28.526981+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=28, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-05T13:13:04.159129+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=13, tm_sec=4, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-05T13:11:28.526981+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=28, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-05T13:12:50.064199+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=12, tm_sec=50, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-05T13:11:28.526981+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=28, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-05T13:12:51.179242+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=12, tm_sec=51, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.505 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-05T13:13:06.992813+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=13, tm_sec=6, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.506 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-05T13:11:28.526981+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=28, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.506 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-05T13:13:11.658445+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=13, tm_sec=11, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.506 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-05T13:11:28.526981+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=28, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.506 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-05T13:13:05.462886+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=13, tm_sec=5, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.506 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-05T13:11:28.526981+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=28, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.506 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-05T13:11:28.526981+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=11, tm_sec=28, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.506 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-05T13:12:59.497416+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=12, tm_sec=59, tm_wday=3, tm_yday=249, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=5, tm_hour=13, tm_min=26, tm_sec=35, tm_wday=3, tm_yday=249, tm_isdst=0) 2024-09-05T13:26:38.506 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-05T13:26:58.508 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-05T13:26:58.799 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-09-05T13:26:58.799 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-09-05T13:26:58.811 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-09-05T13:26:57.793552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291188,"num_write_kb":493971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53499,"num_bytes_recovered":33386515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160874,"ondisk_log_size":160874,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1455836,"kb_used_data":58604,"kb_used_omap":12,"kb_used_meta":1397171,"kb_avail":281659684,"statfs":{"total":289910292480,"available":288419516416,"internally_reserved":0,"allocated":60010496,"data_stored":58886565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13306,"internal_metadata":1430703110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002650"},"pg_stats":[{"pgid":"2.f","version":"47'20972","reported_seq":29953,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:40.461214+0000","last_change":"2024-09-05T13:26:40.461214+0000","last_active":"2024-09-05T13:26:40.461214+0000","last_peered":"2024-09-05T13:26:40.461214+0000","last_clean":"2024-09-05T13:26:40.461214+0000","last_became_active":"2024-09-05T13:24:03.265101+0000","last_became_peered":"2024-09-05T13:24:03.265101+0000","last_unstale":"2024-09-05T13:26:40.461214+0000","last_undegraded":"2024-09-05T13:26:40.461214+0000","last_fullsized":"2024-09-05T13:26:40.461214+0000","mapping_epoch":54,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20972","last_scrub_stamp":"2024-09-05T13:26:40.461110+0000","last_deep_scrub":"47'20972","last_deep_scrub_stamp":"2024-09-05T13:26:40.461110+0000","last_clean_scrub_stamp":"2024-09-05T13:26:40.461110+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:01:40.560252+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20972,"num_write_kb":41944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20754","reported_seq":20915,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:44.021509+0000","last_change":"2024-09-05T13:26:44.021509+0000","last_active":"2024-09-05T13:26:44.021509+0000","last_peered":"2024-09-05T13:26:44.021509+0000","last_clean":"2024-09-05T13:26:44.021509+0000","last_became_active":"2024-09-05T13:12:25.166397+0000","last_became_peered":"2024-09-05T13:12:25.166397+0000","last_unstale":"2024-09-05T13:26:44.021509+0000","last_undegraded":"2024-09-05T13:26:44.021509+0000","last_fullsized":"2024-09-05T13:26:44.021509+0000","mapping_epoch":35,"log_start":"45'10700","ondisk_log_start":"45'10700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20754","last_scrub_stamp":"2024-09-05T13:26:44.021345+0000","last_deep_scrub":"47'20754","last_deep_scrub_stamp":"2024-09-05T13:26:44.021345+0000","last_clean_scrub_stamp":"2024-09-05T13:26:44.021345+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:04:38.924482+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18764,"num_write_kb":33560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'21196","reported_seq":28630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:50.438013+0000","last_change":"2024-09-05T13:26:50.438013+0000","last_active":"2024-09-05T13:26:50.438013+0000","last_peered":"2024-09-05T13:26:50.438013+0000","last_clean":"2024-09-05T13:26:50.438013+0000","last_became_active":"2024-09-05T13:22:50.899816+0000","last_became_peered":"2024-09-05T13:22:50.899816+0000","last_unstale":"2024-09-05T13:26:50.438013+0000","last_undegraded":"2024-09-05T13:26:50.438013+0000","last_fullsized":"2024-09-05T13:26:50.438013+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'21196","last_scrub_stamp":"2024-09-05T13:26:50.437913+0000","last_deep_scrub":"47'21196","last_deep_scrub_stamp":"2024-09-05T13:26:50.437913+0000","last_clean_scrub_stamp":"2024-09-05T13:26:50.437913+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:46:24.111283+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19007,"num_write_kb":33641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4493,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"47'21026","reported_seq":21152,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:42.995433+0000","last_change":"2024-09-05T13:26:42.995433+0000","last_active":"2024-09-05T13:26:42.995433+0000","last_peered":"2024-09-05T13:26:42.995433+0000","last_clean":"2024-09-05T13:26:42.995433+0000","last_became_active":"2024-09-05T13:12:25.167001+0000","last_became_peered":"2024-09-05T13:12:25.167001+0000","last_unstale":"2024-09-05T13:26:42.995433+0000","last_undegraded":"2024-09-05T13:26:42.995433+0000","last_fullsized":"2024-09-05T13:26:42.995433+0000","mapping_epoch":35,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'21026","last_scrub_stamp":"2024-09-05T13:26:42.995309+0000","last_deep_scrub":"47'21026","last_deep_scrub_stamp":"2024-09-05T13:26:42.995309+0000","last_clean_scrub_stamp":"2024-09-05T13:26:42.995309+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:11:41.052985+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21026,"num_write_kb":42052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'20883","reported_seq":21074,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:39.460161+0000","last_change":"2024-09-05T13:26:39.460161+0000","last_active":"2024-09-05T13:26:39.460161+0000","last_peered":"2024-09-05T13:26:39.460161+0000","last_clean":"2024-09-05T13:26:39.460161+0000","last_became_active":"2024-09-05T13:12:24.165453+0000","last_became_peered":"2024-09-05T13:12:24.165453+0000","last_unstale":"2024-09-05T13:26:39.460161+0000","last_undegraded":"2024-09-05T13:26:39.460161+0000","last_fullsized":"2024-09-05T13:26:39.460161+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20883","last_scrub_stamp":"2024-09-05T13:26:39.460035+0000","last_deep_scrub":"47'20883","last_deep_scrub_stamp":"2024-09-05T13:26:39.460035+0000","last_clean_scrub_stamp":"2024-09-05T13:26:39.460035+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T01:17:18.154751+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17423,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'21155","reported_seq":21318,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:48.437501+0000","last_change":"2024-09-05T13:26:48.437501+0000","last_active":"2024-09-05T13:26:48.437501+0000","last_peered":"2024-09-05T13:26:48.437501+0000","last_clean":"2024-09-05T13:26:48.437501+0000","last_became_active":"2024-09-05T13:26:27.952269+0000","last_became_peered":"2024-09-05T13:26:27.952269+0000","last_unstale":"2024-09-05T13:26:48.437501+0000","last_undegraded":"2024-09-05T13:26:48.437501+0000","last_fullsized":"2024-09-05T13:26:48.437501+0000","mapping_epoch":66,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'21155","last_scrub_stamp":"2024-09-05T13:26:48.437405+0000","last_deep_scrub":"47'21155","last_deep_scrub_stamp":"2024-09-05T13:26:48.437405+0000","last_clean_scrub_stamp":"2024-09-05T13:26:48.437405+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:11.020374+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17610,"num_write_kb":28140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4453,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"47'20628","reported_seq":29711,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:49.414900+0000","last_change":"2024-09-05T13:26:49.414900+0000","last_active":"2024-09-05T13:26:49.414900+0000","last_peered":"2024-09-05T13:26:49.414900+0000","last_clean":"2024-09-05T13:26:49.414900+0000","last_became_active":"2024-09-05T13:19:36.111261+0000","last_became_peered":"2024-09-05T13:19:36.111261+0000","last_unstale":"2024-09-05T13:26:49.414900+0000","last_undegraded":"2024-09-05T13:26:49.414900+0000","last_fullsized":"2024-09-05T13:26:49.414900+0000","mapping_epoch":46,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20628","last_scrub_stamp":"2024-09-05T13:26:49.414785+0000","last_deep_scrub":"47'20628","last_deep_scrub_stamp":"2024-09-05T13:26:49.414785+0000","last_clean_scrub_stamp":"2024-09-05T13:26:49.414785+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:02:33.673310+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16947,"num_write_kb":26544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4438,"num_bytes_recovered":15405056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"47'21104","reported_seq":21292,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:41.492204+0000","last_change":"2024-09-05T13:26:41.492204+0000","last_active":"2024-09-05T13:26:41.492204+0000","last_peered":"2024-09-05T13:26:41.492204+0000","last_clean":"2024-09-05T13:26:41.492204+0000","last_became_active":"2024-09-05T13:12:24.163468+0000","last_became_peered":"2024-09-05T13:12:24.163468+0000","last_unstale":"2024-09-05T13:26:41.492204+0000","last_undegraded":"2024-09-05T13:26:41.492204+0000","last_fullsized":"2024-09-05T13:26:41.492204+0000","mapping_epoch":34,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'21104","last_scrub_stamp":"2024-09-05T13:26:41.492073+0000","last_deep_scrub":"47'21104","last_deep_scrub_stamp":"2024-09-05T13:26:41.492073+0000","last_clean_scrub_stamp":"2024-09-05T13:26:41.492073+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:29.631276+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17714,"num_write_kb":28652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:38.534944+0000","last_change":"2024-09-05T13:26:38.534944+0000","last_active":"2024-09-05T13:26:38.534944+0000","last_peered":"2024-09-05T13:26:38.534944+0000","last_clean":"2024-09-05T13:26:38.534944+0000","last_became_active":"2024-09-05T13:13:17.911955+0000","last_became_peered":"2024-09-05T13:13:17.911955+0000","last_unstale":"2024-09-05T13:26:38.534944+0000","last_undegraded":"2024-09-05T13:26:38.534944+0000","last_fullsized":"2024-09-05T13:26:38.534944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:26:38.534881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:26:38.534881+0000","last_clean_scrub_stamp":"2024-09-05T13:26:38.534881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:12:55.536792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070280899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'21093","reported_seq":21283,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:42.532205+0000","last_change":"2024-09-05T13:26:42.532205+0000","last_active":"2024-09-05T13:26:42.532205+0000","last_peered":"2024-09-05T13:26:42.532205+0000","last_clean":"2024-09-05T13:26:42.532205+0000","last_became_active":"2024-09-05T13:23:41.208505+0000","last_became_peered":"2024-09-05T13:23:41.208505+0000","last_unstale":"2024-09-05T13:26:42.532205+0000","last_undegraded":"2024-09-05T13:26:42.532205+0000","last_fullsized":"2024-09-05T13:26:42.532205+0000","mapping_epoch":52,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'21093","last_scrub_stamp":"2024-09-05T13:26:42.532076+0000","last_deep_scrub":"47'21093","last_deep_scrub_stamp":"2024-09-05T13:26:42.532076+0000","last_clean_scrub_stamp":"2024-09-05T13:26:42.532076+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:12:48.449650+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18747,"num_write_kb":32816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4455,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:39.080280+0000","last_change":"2024-09-05T13:26:39.080232+0000","last_active":"2024-09-05T13:26:39.080280+0000","last_peered":"2024-09-05T13:26:39.080280+0000","last_clean":"2024-09-05T13:26:39.080280+0000","last_became_active":"2024-09-05T13:12:25.165024+0000","last_became_peered":"2024-09-05T13:12:25.165024+0000","last_unstale":"2024-09-05T13:26:39.080280+0000","last_undegraded":"2024-09-05T13:26:39.080280+0000","last_fullsized":"2024-09-05T13:26:39.080280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:26:39.080167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:26:39.080167+0000","last_clean_scrub_stamp":"2024-09-05T13:26:39.080167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:23:09.262592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'20910","reported_seq":30025,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:38.436031+0000","last_change":"2024-09-05T13:26:38.435945+0000","last_active":"2024-09-05T13:26:38.436031+0000","last_peered":"2024-09-05T13:26:38.436031+0000","last_clean":"2024-09-05T13:26:38.436031+0000","last_became_active":"2024-09-05T13:24:49.385216+0000","last_became_peered":"2024-09-05T13:24:49.385216+0000","last_unstale":"2024-09-05T13:26:38.436031+0000","last_undegraded":"2024-09-05T13:26:38.436031+0000","last_fullsized":"2024-09-05T13:26:38.436031+0000","mapping_epoch":58,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20910","last_scrub_stamp":"2024-09-05T13:26:38.435862+0000","last_deep_scrub":"47'20910","last_deep_scrub_stamp":"2024-09-05T13:26:38.435862+0000","last_clean_scrub_stamp":"2024-09-05T13:26:38.435862+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:41:28.470541+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17277,"num_write_kb":27304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4457,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"67'3","reported_seq":162,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:37.411802+0000","last_change":"2024-09-05T13:26:37.411711+0000","last_active":"2024-09-05T13:26:37.411802+0000","last_peered":"2024-09-05T13:26:37.411802+0000","last_clean":"2024-09-05T13:26:37.411802+0000","last_became_active":"2024-09-05T13:13:18.016713+0000","last_became_peered":"2024-09-05T13:13:18.016713+0000","last_unstale":"2024-09-05T13:26:37.411802+0000","last_undegraded":"2024-09-05T13:26:37.411802+0000","last_fullsized":"2024-09-05T13:26:37.411802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'3","last_scrub_stamp":"2024-09-05T13:26:37.411607+0000","last_deep_scrub":"67'3","last_deep_scrub_stamp":"2024-09-05T13:26:37.411607+0000","last_clean_scrub_stamp":"2024-09-05T13:26:37.411607+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:52:04.916179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00059730700000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20911","reported_seq":29869,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:42.005356+0000","last_change":"2024-09-05T13:26:42.005301+0000","last_active":"2024-09-05T13:26:42.005356+0000","last_peered":"2024-09-05T13:26:42.005356+0000","last_clean":"2024-09-05T13:26:42.005356+0000","last_became_active":"2024-09-05T13:16:09.202095+0000","last_became_peered":"2024-09-05T13:16:09.202095+0000","last_unstale":"2024-09-05T13:26:42.005356+0000","last_undegraded":"2024-09-05T13:26:42.005356+0000","last_fullsized":"2024-09-05T13:26:42.005356+0000","mapping_epoch":44,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20911","last_scrub_stamp":"2024-09-05T13:26:42.005085+0000","last_deep_scrub":"47'20911","last_deep_scrub_stamp":"2024-09-05T13:26:42.005085+0000","last_clean_scrub_stamp":"2024-09-05T13:26:42.005085+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:02:29.182198+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18506,"num_write_kb":32212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":17969152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"67'2","reported_seq":137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:39.510820+0000","last_change":"2024-09-05T13:26:39.510820+0000","last_active":"2024-09-05T13:26:39.510820+0000","last_peered":"2024-09-05T13:26:39.510820+0000","last_clean":"2024-09-05T13:26:39.510820+0000","last_became_active":"2024-09-05T13:13:17.975183+0000","last_became_peered":"2024-09-05T13:13:17.975183+0000","last_unstale":"2024-09-05T13:26:39.510820+0000","last_undegraded":"2024-09-05T13:26:39.510820+0000","last_fullsized":"2024-09-05T13:26:39.510820+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'2","last_scrub_stamp":"2024-09-05T13:26:39.510743+0000","last_deep_scrub":"67'2","last_deep_scrub_stamp":"2024-09-05T13:26:39.510743+0000","last_clean_scrub_stamp":"2024-09-05T13:26:39.510743+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:02:24.937210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013300160000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20582","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:41.049309+0000","last_change":"2024-09-05T13:26:41.049309+0000","last_active":"2024-09-05T13:26:41.049309+0000","last_peered":"2024-09-05T13:26:41.049309+0000","last_clean":"2024-09-05T13:26:41.049309+0000","last_became_active":"2024-09-05T13:24:22.325081+0000","last_became_peered":"2024-09-05T13:24:22.325081+0000","last_unstale":"2024-09-05T13:26:41.049309+0000","last_undegraded":"2024-09-05T13:26:41.049309+0000","last_fullsized":"2024-09-05T13:26:41.049309+0000","mapping_epoch":56,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20582","last_scrub_stamp":"2024-09-05T13:26:41.049212+0000","last_deep_scrub":"47'20582","last_deep_scrub_stamp":"2024-09-05T13:26:41.049212+0000","last_clean_scrub_stamp":"2024-09-05T13:26:41.049212+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:30:05.953093+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18091,"num_write_kb":31212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20878","reported_seq":21043,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:47.427576+0000","last_change":"2024-09-05T13:26:47.427576+0000","last_active":"2024-09-05T13:26:47.427576+0000","last_peered":"2024-09-05T13:26:47.427576+0000","last_clean":"2024-09-05T13:26:47.427576+0000","last_became_active":"2024-09-05T13:25:57.882414+0000","last_became_peered":"2024-09-05T13:25:57.882414+0000","last_unstale":"2024-09-05T13:26:47.427576+0000","last_undegraded":"2024-09-05T13:26:47.427576+0000","last_fullsized":"2024-09-05T13:26:47.427576+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'20878","last_scrub_stamp":"2024-09-05T13:26:47.427467+0000","last_deep_scrub":"47'20878","last_deep_scrub_stamp":"2024-09-05T13:26:47.427467+0000","last_clean_scrub_stamp":"2024-09-05T13:26:47.427467+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:42:06.455432+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17029,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:40.511847+0000","last_change":"2024-09-05T13:26:40.511847+0000","last_active":"2024-09-05T13:26:40.511847+0000","last_peered":"2024-09-05T13:26:40.511847+0000","last_clean":"2024-09-05T13:26:40.511847+0000","last_became_active":"2024-09-05T13:13:18.105232+0000","last_became_peered":"2024-09-05T13:13:18.105232+0000","last_unstale":"2024-09-05T13:26:40.511847+0000","last_undegraded":"2024-09-05T13:26:40.511847+0000","last_fullsized":"2024-09-05T13:26:40.511847+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:26:40.511738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:26:40.511738+0000","last_clean_scrub_stamp":"2024-09-05T13:26:40.511738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:43:30.757749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014579510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'21104","reported_seq":21327,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:45.464743+0000","last_change":"2024-09-05T13:26:45.464743+0000","last_active":"2024-09-05T13:26:45.464743+0000","last_peered":"2024-09-05T13:26:45.464743+0000","last_clean":"2024-09-05T13:26:45.464743+0000","last_became_active":"2024-09-05T13:25:28.512014+0000","last_became_peered":"2024-09-05T13:25:28.512014+0000","last_unstale":"2024-09-05T13:26:45.464743+0000","last_undegraded":"2024-09-05T13:26:45.464743+0000","last_fullsized":"2024-09-05T13:26:45.464743+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'21104","last_scrub_stamp":"2024-09-05T13:26:45.464644+0000","last_deep_scrub":"47'21104","last_deep_scrub_stamp":"2024-09-05T13:26:45.464644+0000","last_clean_scrub_stamp":"2024-09-05T13:26:45.464644+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:45:12.588109+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17003,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4447,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:40.083018+0000","last_change":"2024-09-05T13:26:40.082766+0000","last_active":"2024-09-05T13:26:40.083018+0000","last_peered":"2024-09-05T13:26:40.083018+0000","last_clean":"2024-09-05T13:26:40.083018+0000","last_became_active":"2024-09-05T13:12:25.166378+0000","last_became_peered":"2024-09-05T13:12:25.166378+0000","last_unstale":"2024-09-05T13:26:40.083018+0000","last_undegraded":"2024-09-05T13:26:40.083018+0000","last_fullsized":"2024-09-05T13:26:40.083018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:26:40.082673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:26:40.082673+0000","last_clean_scrub_stamp":"2024-09-05T13:26:40.082673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:37:18.513216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'21080","reported_seq":21280,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:46.425216+0000","last_change":"2024-09-05T13:26:46.425216+0000","last_active":"2024-09-05T13:26:46.425216+0000","last_peered":"2024-09-05T13:26:46.425216+0000","last_clean":"2024-09-05T13:26:46.425216+0000","last_became_active":"2024-09-05T13:23:22.081272+0000","last_became_peered":"2024-09-05T13:23:22.081272+0000","last_unstale":"2024-09-05T13:26:46.425216+0000","last_undegraded":"2024-09-05T13:26:46.425216+0000","last_fullsized":"2024-09-05T13:26:46.425216+0000","mapping_epoch":50,"log_start":"45'11000","ondisk_log_start":"45'11000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'21080","last_scrub_stamp":"2024-09-05T13:26:46.425123+0000","last_deep_scrub":"47'21080","last_deep_scrub_stamp":"2024-09-05T13:26:46.425123+0000","last_clean_scrub_stamp":"2024-09-05T13:26:46.425123+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:13:08.041288+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:41.497110+0000","last_change":"2024-09-05T13:26:41.497110+0000","last_active":"2024-09-05T13:26:41.497110+0000","last_peered":"2024-09-05T13:26:41.497110+0000","last_clean":"2024-09-05T13:26:41.497110+0000","last_became_active":"2024-09-05T13:13:18.035794+0000","last_became_peered":"2024-09-05T13:13:18.035794+0000","last_unstale":"2024-09-05T13:26:41.497110+0000","last_undegraded":"2024-09-05T13:26:41.497110+0000","last_fullsized":"2024-09-05T13:26:41.497110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:26:41.497025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:26:41.497025+0000","last_clean_scrub_stamp":"2024-09-05T13:26:41.497025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:20:53.003353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069656799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":131,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:44.483799+0000","last_change":"2024-09-05T13:26:44.483799+0000","last_active":"2024-09-05T13:26:44.483799+0000","last_peered":"2024-09-05T13:26:44.483799+0000","last_clean":"2024-09-05T13:26:44.483799+0000","last_became_active":"2024-09-05T13:13:17.905803+0000","last_became_peered":"2024-09-05T13:13:17.905803+0000","last_unstale":"2024-09-05T13:26:44.483799+0000","last_undegraded":"2024-09-05T13:26:44.483799+0000","last_fullsized":"2024-09-05T13:26:44.483799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T13:26:44.483715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T13:26:44.483715+0000","last_clean_scrub_stamp":"2024-09-05T13:26:44.483715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:35:20.543455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010267259999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"67'21193","reported_seq":21373,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-05T13:26:43.493396+0000","last_change":"2024-09-05T13:26:43.493396+0000","last_active":"2024-09-05T13:26:43.493396+0000","last_peered":"2024-09-05T13:26:43.493396+0000","last_clean":"2024-09-05T13:26:43.493396+0000","last_became_active":"2024-09-05T13:25:10.447760+0000","last_became_peered":"2024-09-05T13:25:10.447760+0000","last_unstale":"2024-09-05T13:26:43.493396+0000","last_undegraded":"2024-09-05T13:26:43.493396+0000","last_fullsized":"2024-09-05T13:26:43.493396+0000","mapping_epoch":60,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'21193","last_scrub_stamp":"2024-09-05T13:26:43.493301+0000","last_deep_scrub":"67'21193","last_deep_scrub_stamp":"2024-09-05T13:26:43.493301+0000","last_clean_scrub_stamp":"2024-09-05T13:26:43.493301+0000","objects_scrubbed":1,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:28:52.669641+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17860,"num_write_kb":29080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4568,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":291186,"num_write_kb":493969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53498,"num_bytes_recovered":33386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57839616,"data_stored":57832390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986355,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":383472,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93988368,"statfs":{"total":96636764160,"available":96244088832,"internally_reserved":0,"allocated":557056,"data_stored":187539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4435,"internal_metadata":392097453},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552820,"kb_used_data":57508,"kb_used_omap":4,"kb_used_meta":495291,"kb_avail":93819020,"statfs":{"total":96636764160,"available":96070676480,"internally_reserved":0,"allocated":58888192,"data_stored":58510985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":507178666},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519544,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852296,"statfs":{"total":96636764160,"available":96104751104,"internally_reserved":0,"allocated":565248,"data_stored":188041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57835520,"data_stored":57831907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T13:26:58.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-05T13:26:59.111 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-05T13:26:59.111 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-05T13:26:59.112 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-05T13:26:59.112 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:27:00.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-05T13:27:05.213 INFO:tasks.ceph.osd.0:Stopped 2024-09-05T13:27:05.214 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-05T13:27:05.214 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:27:05.293 INFO:tasks.ceph.osd.1:Stopped 2024-09-05T13:27:05.294 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-05T13:27:05.294 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:27:05.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-05T13:27:05.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-05T13:27:11.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-05T13:27:11.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-05T13:27:11.395 INFO:tasks.ceph.osd.2:Stopped 2024-09-05T13:27:11.396 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-05T13:27:11.396 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-05T13:27:11.396 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:27:11.426 INFO:tasks.ceph.mgr.x:Stopped 2024-09-05T13:27:11.426 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-05T13:27:11.427 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-05T13:27:11.427 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:27:11.481 INFO:tasks.ceph.mon.a:Stopped 2024-09-05T13:27:11.481 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-05T13:27:11.481 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:27:11.534 INFO:tasks.ceph.mon.b:Stopped 2024-09-05T13:27:11.535 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-05T13:27:11.535 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T13:27:11.588 INFO:tasks.ceph.mon.c:Stopped 2024-09-05T13:27:11.588 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-05T13:27:11.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-05T13:27:11.804 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:27:11.805 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-05T13:27:12.456 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:27:12.456 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-05T13:27:12.573 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:27:12.573 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-05T13:27:12.644 INFO:tasks.ceph:Archiving mon data... 2024-09-05T13:27:12.645 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478/data/mon.a.tgz 2024-09-05T13:27:12.646 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-09-05T13:27:12.673 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.toDUPcJ4ZW 2024-09-05T13:27:12.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.toDUPcJ4ZW 2024-09-05T13:27:12.809 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.toDUPcJ4ZW 2024-09-05T13:27:12.954 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.toDUPcJ4ZW is 298KB 2024-09-05T13:27:13.049 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.toDUPcJ4ZW 2024-09-05T13:27:13.063 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478/data/mon.b.tgz 2024-09-05T13:27:13.064 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-09-05T13:27:13.119 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.2r7OHpRcqz 2024-09-05T13:27:13.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2r7OHpRcqz 2024-09-05T13:27:13.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.2r7OHpRcqz 2024-09-05T13:27:13.353 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.2r7OHpRcqz is 299KB 2024-09-05T13:27:13.947 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.2r7OHpRcqz 2024-09-05T13:27:13.964 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478/data/mon.c.tgz 2024-09-05T13:27:13.965 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-09-05T13:27:14.021 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.WFUq5Kr5fN 2024-09-05T13:27:14.022 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WFUq5Kr5fN 2024-09-05T13:27:14.140 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.WFUq5Kr5fN 2024-09-05T13:27:14.227 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.WFUq5Kr5fN is 299KB 2024-09-05T13:27:14.389 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.WFUq5Kr5fN 2024-09-05T13:27:14.405 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-05T13:27:14.406 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-05T13:27:14.506 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-05T13:27:14.506 INFO:tasks.ceph:Archiving crash dumps... 2024-09-05T13:27:14.508 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478/remote/smithi086/crash 2024-09-05T13:27:14.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-05T13:27:14.570 INFO:tasks.ceph:Compressing logs... 2024-09-05T13:27:14.571 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T13:27:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-05T13:27:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40469.log 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49829.log 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46699.log 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40469.log.gz 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51845.log 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49829.log.gz 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73585.log 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46699.log.gz 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50333.log 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51845.log.gz 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38003.log 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73585.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73585.log.gz 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44901.log 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50333.log.gz 2024-09-05T13:27:14.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66863.log 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38003.log.gz 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72787.log 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44901.log.gz 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46144.log 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66863.log.gz 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51383.log 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72787.log.gz 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40210.log 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46144.log.gz 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45447.log 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51383.log.gz 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41878.log 2024-09-05T13:27:14.931 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40210.log.gz 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69300.log 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45447.log.gz 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49661.log 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41878.log.gz 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43328.log 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69300.log.gz 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64215.log 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49661.log.gz 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57853.log 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43328.log.gz 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73291.log 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64215.log.gz 2024-09-05T13:27:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46070.log 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57853.log.gz 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59084.log 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73291.log.gz 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56255.log 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46070.log.gz 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49199.log 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59084.log.gz 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71527.log 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56255.log.gz 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50921.log 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49199.log.gz 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70561.log/var/log/ceph/ceph-client.admin.71527.log: 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71527.log.gz 2024-09-05T13:27:14.933 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55289.log 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50921.log.gz 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38502.log 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70561.log.gz 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37746.log 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55289.log.gz 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45615.log 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38502.log.gz 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.37746.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49325.log 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37746.log.gz 2024-09-05T13:27:14.934 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68880.log 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45615.log.gz 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62409.log 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49325.log.gz 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70519.log 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68880.log.gz 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67997.log 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62409.log.gz 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52433.log 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70519.log.gz 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45996.log 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67997.log.gz 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65981.log 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52433.log.gz 2024-09-05T13:27:14.935 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45996.log.gz 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40136.log 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72283.log 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65981.log.gz 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60137.log 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40136.log.gz 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64761.log 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72283.log.gz 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48233.log 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60137.log.gz 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51173.log 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64761.log.gz 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40617.log 2024-09-05T13:27:14.936 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48233.log.gz 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68543.log 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51173.log.gz 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48695.log 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40617.log.gz 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38757.log 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68543.log.gz 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73796.log 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48695.log.gz 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38757.log.gz 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67367.log 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45848.log 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73796.log.gz 2024-09-05T13:27:14.937 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64089.log 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67367.log.gz 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44355.log 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45848.log.gz 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52853.log 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64089.log.gz 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.44355.log: --verbose -- /var/log/ceph/ceph-client.admin.57685.log 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44355.log.gz 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54659.log 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52853.log.gz 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57685.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57685.log.gz 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69804.log 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73670.log 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54659.log.gz 2024-09-05T13:27:14.938 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66779.log 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69804.log.gz 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46551.log 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73670.log.gz 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73165.log 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66779.log.gz 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66737.log 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46551.log.gz 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53567.log 2024-09-05T13:27:14.939 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73165.log.gz 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69342.log 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66737.log.gz 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69258.log 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53567.log.gz 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59630.log 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69342.log.gz 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67661.log 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69258.log.gz 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49871.log 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59630.log.gz 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37706.log 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67661.log.gz 2024-09-05T13:27:14.940 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65644.log 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49871.log.gz 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59252.log 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37706.log.gz 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60053.log 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65644.log.gz 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59252.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.59252.log.gz /var/log/ceph/ceph-client.admin.40654.log 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38831.log 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60053.log.gz 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40247.log 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40654.log.gz 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72577.log 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38831.log.gz 2024-09-05T13:27:14.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43561.log 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40247.log.gz 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45885.log 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72577.log.gz 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50375.log 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62745.log 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45885.log.gz 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67913.log 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50375.log.gz 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69594.log 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62745.log.gz 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57769.log 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67913.log.gz 2024-09-05T13:27:14.942 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69594.log.gz 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57811.log 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57769.log.gz 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57179.log 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57095.log 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57811.log.gz 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50207.log 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57179.log.gz 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51215.log 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57095.log.gz 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69720.log 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50207.log.gz 2024-09-05T13:27:14.943 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51761.log 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51215.log.gz 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59840.log 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69720.log.gz 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39090.log 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51761.log.gz 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62283.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59840.log.gz 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73628.log 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39090.log.gz 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57895.log 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62283.log.gz 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69090.log 2024-09-05T13:27:14.944 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73628.log.gz 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59168.log 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57895.log.gz 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51635.log 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69090.log.gz 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48107.log 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59168.log.gz 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43603.log 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51635.log.gz 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.48107.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73123.log 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48107.log.gz 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65307.log 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43603.log.gz 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52475.log 2024-09-05T13:27:14.945 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73123.log.gz 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46181.log 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65307.log.gz 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44313.log 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52475.log.gz 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71695.log 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46181.log.gz 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65349.log 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44313.log.gz 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49535.log 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71695.log.gz 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41606.log 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65349.log.gz 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58580.log 2024-09-05T13:27:14.946 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49535.log.gz 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41606.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41606.log.gz 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59672.log 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58580.log.gz 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39914.log 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44481.log 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59672.log.gz 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63921.log 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39914.log.gz 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37745.log 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44481.log.gz 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39460.log 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63921.log.gz 2024-09-05T13:27:14.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73543.log 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37745.log.gz 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48527.log 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39460.log.gz 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56801.log 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73543.log.gz 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68207.log 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48527.log.gz 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58285.log 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56801.log.gz 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51803.log 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68207.log.gz 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61316.log 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58285.log.gz 2024-09-05T13:27:14.948 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48443.log 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51803.log.gz 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61190.log 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61316.log.gz 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73838.log 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48443.log.gz 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48191.log 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61190.log.gz 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60391.log 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73838.log.gz 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50543.log 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48191.log.gz 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43244.log 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60391.log.gz 2024-09-05T13:27:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45153.log 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50543.log.gz 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43244.log.gz 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61988.log 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45153.log.gz 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39840.log 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65939.log 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61988.log.gz 2024-09-05T13:27:14.950 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52895.log 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39840.log.gz 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65939.log.gz 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71653.log 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52895.log.gz 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65265.log 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66947.log 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71653.log.gz 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58748.log 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65265.log.gz 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39053.log 2024-09-05T13:27:14.951 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66947.log.gz 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54155.log 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58748.log.gz 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74174.log 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39053.log.gz 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63585.log 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54155.log.gz 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63417.log 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74174.log.gz 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51425.log 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63585.log.gz 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39238.log 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63417.log.gz 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37328.log 2024-09-05T13:27:14.952 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51425.log.gz 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51341.log 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39238.log.gz 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37328.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37328.log.gz 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61610.log 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66275.log 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51341.log.gz 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70393.log 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61610.log.gz 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65517.log 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66275.log.gz 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65475.log 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70393.log.gz 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45959.log 2024-09-05T13:27:14.953 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65517.log.gz 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65475.log.gz 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45959.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45959.log.gz 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60433.log 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71989.log 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gz 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66821.log 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60433.log.gz 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53861.log 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71989.log.gz 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57137.log 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66821.log.gz 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74132.log 2024-09-05T13:27:14.954 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53861.log.gz 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58021.log 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57137.log.gz 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40543.log 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74132.log.gz 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38979.log 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58021.log.gz 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54533.log 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40543.log.gz 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39497.log 2024-09-05T13:27:14.955 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38979.log.gz 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54533.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54533.log.gz 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr: /var/log/ceph/ceph-client.admin.44187.log 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64845.log 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39497.log.gz 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55709.log 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44187.log.gz 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70224.log 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64845.log.gz 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39349.log 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55709.log.gz 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44859.log 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70224.log.gz 2024-09-05T13:27:14.956 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66023.log 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39349.log.gz 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56171.log 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44859.log.gz 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39534.log 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66023.log.gz 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54575.log 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56171.log.gz 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69426.log 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39534.log.gz 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52979.log 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54575.log.gz 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61778.log 2024-09-05T13:27:14.957 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69426.log.gz 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57305.log 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52979.log.gz 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.61778.log: --verbose -- /var/log/ceph/ceph-client.admin.66905.log 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61778.log.gz 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54323.log 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57305.log.gz 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52937.log 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66905.log.gz 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56675.log 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54323.log.gz 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38396.log 2024-09-05T13:27:14.958 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52937.log.gz 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67157.log 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56675.log.gz 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.38396.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.66569.log 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38396.log.gz 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36862.log 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67157.log.gz 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38586.log 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66569.log.gz 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61274.log 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36862.log.gz 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71275.log 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38586.log.gz 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63501.log 2024-09-05T13:27:14.959 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61274.log.gz 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56633.log 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71275.log.gz 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48947.log 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63501.log.gz 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60011.log 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56633.log.gz 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65728.log 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48947.log.gz 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45383.log 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60011.log.gz 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55793.log 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65728.log.gz 2024-09-05T13:27:14.960 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64341.log 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45383.log.gz 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65139.log 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55793.log.gz 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70266.log 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64341.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64341.log.gz 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45321.log 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65139.log.gz 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70266.log.gz 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46810.log 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64005.log 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45321.log.gz 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57937.log 2024-09-05T13:27:14.961 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46810.log.gz 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66485.log 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64005.log.gz 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73501.log 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57937.log.gz 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45111.log 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66485.log.gz 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53945.log 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73501.log.gz 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69216.log 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45111.log.gz 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40099.log 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53945.log.gz 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55499.log 2024-09-05T13:27:14.962 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69216.log.gz 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64803.log 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40099.log.gz 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62367.log 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55499.log.gz 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49367.log 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64803.log.gz 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66989.log 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62367.log.gz 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52727.log 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49367.log.gz 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37494.log 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66989.log.gz 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58063.log 2024-09-05T13:27:14.963 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52727.log.gz 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50291.log 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37494.log.gz 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58063.log.gz 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.50291.log: --verbose -- /var/log/ceph/ceph-client.admin.44103.log 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50291.log.gz 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57389.log 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gz 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48863.log 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44103.log.gz 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69678.log 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57389.log.gz 2024-09-05T13:27:14.964 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69888.log 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48863.log.gz 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57601.log 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69678.log.gz 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40432.log 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69888.log.gz 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47939.log 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57601.log.gz 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56465.log 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40432.log.gz 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54449.log 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47939.log.gz 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39766.log 2024-09-05T13:27:14.965 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56465.log.gz 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36864.log 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54449.log.gz 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56423.log 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39766.log.gz 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43833.log 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36864.log.gz 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71443.log 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56423.log.gz 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39571.log 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43833.log.gz 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44943.log 2024-09-05T13:27:14.966 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71443.log.gz 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43832.log 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39571.log.gz 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72115.log 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44943.log.gz 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70140.log 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43832.log.gz 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65602.log 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72115.log.gz 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49241.log 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70140.log.gz 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32719.log 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65602.log.gz 2024-09-05T13:27:14.967 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48275.log 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49241.log.gz 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72535.log 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32719.log.gz 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54869.log 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48275.log.gz 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59126.log 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72535.log.gz 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72367.log 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54869.log.gz 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60180.log 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59126.log.gz 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54911.log 2024-09-05T13:27:14.968 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72367.log.gz 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49409.log 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60180.log.gz 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62829.log 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54911.log.gz 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56297.log 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49409.log.gz 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69930.log 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62829.log.gz 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73039.log 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56297.log.gz 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70014.log 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69930.log.gz 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67073.log 2024-09-05T13:27:14.969 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73039.log.gz 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60601.log 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70014.log.gz 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58453.log 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67073.log.gz 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39127.log 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60601.log.gz 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68333.log 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58453.log.gz 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58790.log 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39127.log.gz 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53063.log 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68333.log.gz 2024-09-05T13:27:14.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66611.log 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58790.log.gz 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60307.log 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53063.log.gz 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72241.log 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66611.log.gz 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60307.log.gz 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68796.log 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41961.log 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72241.log.gz 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68796.log.gz 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44397.log 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71317.log 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41961.log.gz 2024-09-05T13:27:14.971 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37620.log 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44397.log.gz 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69552.log 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71317.log.gz 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45237.log 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37620.log.gz 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45195.log 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69552.log.gz 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43393.log 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45237.log.gz 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38045.log 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45195.log.gz 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72829.log 2024-09-05T13:27:14.972 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43393.log.gz 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46736.log 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38045.log.gz 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51257.log 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72829.log.gz 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69006.log 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46736.log.gz 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57517.log 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51257.log.gz 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69006.log.gz 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72913.log 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67241.log 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57517.log.gz 2024-09-05T13:27:14.973 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62913.log 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72913.log.gz 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48149.log 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67241.log.gz 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60265.log 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62913.log.gz 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46662.log 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48149.log.gz 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44229.log 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60265.log.gz 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58190.log 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46662.log.gz 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63669.log 2024-09-05T13:27:14.974 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44229.log.gz 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51131.log 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58190.log.gz 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63669.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27823.log 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63669.log.gz 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36698.log 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51131.log.gz 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.tmp-client.admin.27823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59588.log 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27823.log.gz 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39729.log 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36698.log.gz 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60643.log 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59588.log.gz 2024-09-05T13:27:14.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39729.log.gz 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44565.log 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68501.log 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60643.log.gz 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73417.log 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44565.log.gz 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54407.log 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68501.log.gz 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36656.log 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73417.log.gz 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54407.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54407.log.gz 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68711.log 2024-09-05T13:27:14.976 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36656.log.gz 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68039.log 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55541.log 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68711.log.gz 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52097.log 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68039.log.gz 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66317.log 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55541.log.gz 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64971.log 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52097.log.gz 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36782.log 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66317.log.gz 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37412.log 2024-09-05T13:27:14.977 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64971.log.gz 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65560.log 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36782.log.gz 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52811.log 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37412.log.gz 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58832.log 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65560.log.gz 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40321.log 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52811.log.gz 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44061.log 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58832.log.gz 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62955.log 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40321.log.gz 2024-09-05T13:27:14.978 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58327.log 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44061.log.gz 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36863.log 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62955.log.gz 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58327.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58327.log.gz 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65097.log 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49745.log 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36863.log.gz 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65097.log.gz 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38942.log 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.49745.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63291.log 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49745.log.gz 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49955.log 2024-09-05T13:27:14.979 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38942.log.gz 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67031.log 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63291.log.gz 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60685.log 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49955.log.gz 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67031.log.gz 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55163.log 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54785.log 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60685.log.gz 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32540.log 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55163.log.gz 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48065.log 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54785.log.gz 2024-09-05T13:27:14.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39803.log 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32540.log.gz 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48779.log 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48065.log.gz 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38395.log 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39803.log.gz 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55415.log 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48779.log.gz 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38395.log.gz 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44145.log 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55415.log.gz 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59420.log 2024-09-05T13:27:14.981 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50417.log 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44145.log.gz 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71863.log 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59420.log.gz 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61105.log 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50417.log.gz 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71863.log.gz 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65686.log 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61105.log.gz 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60853.log 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gz 2024-09-05T13:27:14.982 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39201.log 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65686.log.gz 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58874.log 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60853.log.gz 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39201.log.gz 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71191.log 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58874.log.gz 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61147.log 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38228.log 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71191.log.gz 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67829.log 2024-09-05T13:27:14.983 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61147.log.gz 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38544.log 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38228.log.gz 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38683.log 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67829.log.gz 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70729.log 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38544.log.gz 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66191.log 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38683.log.gz 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58622.log 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70729.log.gz 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58916.log 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66191.log.gz 2024-09-05T13:27:14.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48317.log 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58622.log.gz 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37832.log 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58916.log.gz 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47981.log 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48317.log.gz 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62199.log 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37832.log.gz 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50459.log 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47981.log.gz 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53399.log 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62199.log.gz 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52643.log 2024-09-05T13:27:14.985 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50459.log.gz 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53189.log 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53399.log.gz 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67745.log 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52643.log.gz 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51005.log 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53189.log.gz 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66653.log 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67745.log.gz 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51005.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51005.log.gz 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72157.log 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71611.log 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66653.log.gz 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57221.log 2024-09-05T13:27:14.986 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72157.log.gz 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71611.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.71611.log.gz --verbose 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.68081.log 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53525.log 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57221.log.gz 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46033.log 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68081.log.gz 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61862.log 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53525.log.gz 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68922.log 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46033.log.gz 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47855.log 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61862.log.gz 2024-09-05T13:27:14.987 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68922.log.gz 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46255.log 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47855.log.gz 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36524.log 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46255.log.gz 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64299.log 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54113.log 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36524.log.gz 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62577.log 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64299.log.gz 2024-09-05T13:27:14.988 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62451.log 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54113.log.gz 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.62577.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.74342.log 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62577.log.gz 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55331.log 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62451.log.gz 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67283.log 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74342.log.gz 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71905.log 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55331.log.gz 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49787.log 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67283.log.gz 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56927.log 2024-09-05T13:27:14.989 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71905.log.gz 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59756.log 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49787.log.gz 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61232.log 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56927.log.gz 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71737.log 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59756.log.gz 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55877.log 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61232.log.gz 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67451.log 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71737.log.gz 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58242.log 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55877.log.gz 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70603.log 2024-09-05T13:27:14.990 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67451.log.gz 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39164.log 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58242.log.gz 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70603.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70603.log.gz -5 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63711.log 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51719.log 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39164.log.gz 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49493.log 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63711.log.gz 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51719.log.gz 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64635.log 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61484.log 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49493.log.gz 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58706.log 2024-09-05T13:27:14.991 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64635.log.gz 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32421.log 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61484.log.gz 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55037.log 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58706.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58706.log.gz 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60223.log 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32421.log.gz 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49157.log 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55037.log.gz 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62072.log 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60223.log.gz 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56381.log 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49157.log.gz 2024-09-05T13:27:14.992 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45382.log 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62072.log.gz 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49115.log 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56381.log.gz 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67199.log 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45382.log.gz 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49115.log.gz 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61526.log 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63795.log 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67199.log.gz 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53021.log 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61526.log.gz 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36580.log 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63795.log.gz 2024-09-05T13:27:14.993 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58369.log 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53021.log.gz 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65770.log 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36580.log.gz 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67577.log 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58369.log.gz 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70477.log 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65770.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65770.log.gz 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56213.log 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67577.log.gz 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70477.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70477.log.gz 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63459.log 2024-09-05T13:27:14.994 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56213.log.gz 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64425.log 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37286.log 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63459.log.gz 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61442.log 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64425.log.gz 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56843.log 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37286.log.gz 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68123.log 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61442.log.gz 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36990.log 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56843.log.gz 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55751.log 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68123.log.gz 2024-09-05T13:27:14.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50711.log 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36990.log.gz 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65223.log 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55751.log.gz 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50711.log.gz 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40062.log 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39312.log 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65223.log.gz 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57475.log 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40062.log.gz 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54827.log 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39312.log.gz 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57727.log 2024-09-05T13:27:14.996 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57475.log.gz 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56591.log 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54827.log.gz 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59714.log 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57727.log.gz 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64593.log 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56591.log.gz 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59462.log 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59714.log.gz 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45027.log 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr: 95.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64593.log.gz 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74258.log 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59462.log.gz 2024-09-05T13:27:14.997 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45027.log: /var/log/ceph/ceph-client.admin.61063.log 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45027.log.gz 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63249.log 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74258.log.gz 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63333.log 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61063.log.gz 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53315.log 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63249.log.gz 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63333.log.gz 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72997.log 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53315.log.gz 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54071.log 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49577.log 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72997.log.gz 2024-09-05T13:27:14.998 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55457.log 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54071.log.gz 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48737.log 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49577.log.gz 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68669.log 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55457.log.gz 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44775.log 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48737.log.gz 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38646.log 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68669.log.gz 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73754.log 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44775.log.gz 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61946.log 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38646.log.gz 2024-09-05T13:27:14.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68417.log 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73754.log.gz 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74006.log 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61946.log.gz 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62535.log 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68417.log.gz 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74006.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.74006.log.gz 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr: /var/log/ceph/ceph-client.admin.58664.log 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61736.log 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62535.log.gz 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51677.log 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58664.log.gz 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43981.log 2024-09-05T13:27:15.000 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61736.log.gz 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53483.log 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51677.log.gz 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57011.log 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43981.log.gz 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53483.log.gz 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59798.log 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40580.log 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57011.log.gz 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48653.log 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59798.log.gz 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39627.log 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40580.log.gz 2024-09-05T13:27:15.001 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39877.log 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48653.log.gz 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61358.log 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39627.log.gz 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61021.log 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39877.log.gz 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38311.log 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61358.log.gz 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67871.log 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61021.log.gz 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68627.log 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38311.log.gz 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54197.log 2024-09-05T13:27:15.002 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67871.log.gz 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54743.log 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68627.log.gz 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36989.log 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54197.log.gz 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42110.log 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54743.log.gz 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61904.log 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36989.log.gz 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62325.log 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42110.log.gz 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61904.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.61904.log.gz -- 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr: /var/log/ceph/ceph-client.admin.70687.log 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51467.log 2024-09-05T13:27:15.003 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62325.log.gz 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65013.log 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70687.log.gz 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70897.log 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51467.log.gz 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65013.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65013.log.gz 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71779.log 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52181.log 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70897.log.gz 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39275.log 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71779.log.gz 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55121.log 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52181.log.gz 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55583.log 2024-09-05T13:27:15.004 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39275.log.gz 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64173.log 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55121.log.gz 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51971.log 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55583.log.gz 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56087.log 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64173.log.gz 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46366.log 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51971.log.gz 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68165.log 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56087.log.gz 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68459.log 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46366.log.gz 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64131.log 2024-09-05T13:27:15.005 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68165.log.gz 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68964.log 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68459.log.gz 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74090.log 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64131.log.gz 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56339.log 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68964.log.gz 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63627.log 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74090.log.gz 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73081.log 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56339.log.gz 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39692.log 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63627.log.gz 2024-09-05T13:27:15.006 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59546.log 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73081.log.gz 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46514.log 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39692.log.gz 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66107.log 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59546.log.gz 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54365.log 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46514.log.gz 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68291.log 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66107.log.gz 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63375.log 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54365.log.gz 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70351.log 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68291.log.gz 2024-09-05T13:27:15.007 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67493.log 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63375.log.gz 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51299.log 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70351.log.gz 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45657.log 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67493.log.gz 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51299.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51299.log.gz 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59042.log 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45657.log.gz 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64509.log 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62871.log 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59042.log.gz 2024-09-05T13:27:15.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43245.log 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64509.log.gz 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39951.log 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62871.log.gz 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58105.log 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43245.log.gz 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32580.log 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39951.log.gz 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64887.log 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58105.log.gz 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62493.log 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32580.log.gz 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43748.log 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64887.log.gz 2024-09-05T13:27:15.009 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63123.log 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62493.log.gz 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60769.log 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43748.log.gz 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51887.log 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63123.log.gz 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60517.log 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60769.log.gz 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68838.log 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51887.log.gz 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70813.log 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60517.log.gz 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32792.log 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68838.log.gz 2024-09-05T13:27:15.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46107.log 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70813.log.gz 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70939.log 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32792.log.gz 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47813.log 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46107.log.gz 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64929.log 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70939.log.gz 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36740.log 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47813.log.gz 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64929.log.gz 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71485.log 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50165.log 2024-09-05T13:27:15.011 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36740.log.gz 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46218.log 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71485.log.gz 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38312.log 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50165.log.gz 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46218.log.gz 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45467.log 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58537.log 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38312.log.gz 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45467.log.gz 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43497.log 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58537.log.gz 2024-09-05T13:27:15.012 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73459.log 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60095.log 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43497.log.gz 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61652.log 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73459.log.gz 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54701.log 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60095.log.gz 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46625.log 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61652.log.gz 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56045.log 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54701.log.gz 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48989.log 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46625.log.gz 2024-09-05T13:27:15.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61820.log 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56045.log.gz 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40025.log 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48989.log.gz 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61820.log.gz 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70308.log 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53819.log 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40025.log.gz 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70308.log.gz 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63081.log 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55667.log 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53819.log.gz 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37076.log 2024-09-05T13:27:15.014 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63081.log.gz 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56717.log 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55667.log.gz 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49703.log 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37076.log.gz 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54281.log 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56717.log.gz 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49703.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49703.log.gz 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57643.log 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54281.log.gz 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40173.log 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57643.log.gz 2024-09-05T13:27:15.015 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69636.log 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73249.log 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40173.log.gz 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55919.log 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69636.log.gz 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49283.log 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73249.log.gz 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48821.log 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55919.log.gz 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59378.log 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49283.log.gz 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67703.log 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48821.log.gz 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71569.log 2024-09-05T13:27:15.016 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59378.log.gz 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72661.log 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67703.log.gz 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63837.log 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71569.log.gz 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72661.log.gz 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49997.log 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.63837.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50795.log 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63837.log.gz 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60559.log 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49997.log.gz 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43749.log 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50795.log.gz 2024-09-05T13:27:15.017 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37916.log 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60559.log.gz 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60475.log 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43749.log.gz 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53147.log 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37916.log.gz 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49451.log 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60475.log.gz 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48401.log 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53147.log.gz 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70182.log 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49451.log.gz 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67409.log 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48401.log.gz 2024-09-05T13:27:15.018 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38609.log 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70182.log.gz 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62619.log 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67409.log.gz 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41793.log 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38609.log.gz 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49619.log 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62619.log.gz 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64257.log 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41793.log.gz 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41710.log 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49619.log.gz 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69132.log 2024-09-05T13:27:15.019 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64257.log.gz 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55373.log 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41710.log.gz 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67955.log 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69132.log.gz 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45531.log 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55373.log.gz 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35331.log 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67955.log.gz 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70645.log 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45531.log.gz 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57347.log 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35331.log.gz 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62156.log 2024-09-05T13:27:15.020 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70645.log.gz 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57347.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57347.log.gz -5 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74300.log 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65854.log 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62156.log.gz 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51509.log 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74300.log.gz 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50501.log 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65854.log.gz 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51509.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.51509.log.gz 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.65433.log 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58147.log 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50501.log.gz 2024-09-05T13:27:15.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64677.log 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65433.log.gz 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37874.log 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58147.log.gz 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73207.log 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64677.log.gz 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44004.log 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37874.log.gz 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46403.log 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73207.log.gz 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44004.log.gz 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46773.log 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46403.log.gz 2024-09-05T13:27:15.022 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54239.log 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48485.log 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46773.log.gz 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69762.log 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54239.log.gz 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48485.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48485.log.gz 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50249.log 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69762.log.gz 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64047.log 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66443.log 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50249.log.gz 2024-09-05T13:27:15.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74384.log 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64047.log.gz 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66443.log.gz 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54617.log 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71107.log 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74384.log.gz 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54617.log.gz 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48023.log 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71107.log.gz 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71065.log 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48023.log.gzgzip 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44271.log 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73712.log 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71065.log.gz 2024-09-05T13:27:15.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44271.log.gz 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70435.log 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58958.log 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73712.log.gz 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70435.log.gz 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41877.log 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59000.log 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58958.log.gz 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69468.log 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41877.log.gz 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50879.log 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59000.log.gz 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69468.log.gz 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-09-05T13:27:15.025 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40358.log 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50879.log.gz 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63039.log 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45069.log 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40358.log.gz 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63039.log.gz 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71359.log 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62997.log 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45069.log.gz 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr: 95.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71359.log.gz 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73880.log 2024-09-05T13:27:15.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41794.log 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62997.log.gz 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57053.log 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73880.log.gz 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48611.log 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41794.log.gz 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68754.log 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57053.log.gz 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48611.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48611.log.gz -5 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53273.log 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40284.log 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68754.log.gz 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70098.log 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53273.log.gz 2024-09-05T13:27:15.027 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65391.log 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40284.log.gz 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43916.log 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70098.log.gz 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52685.log 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65391.log.gz 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62241.log 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43916.log.gz 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52391.log 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52685.log.gz 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60979.log 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62241.log.gz 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37116.log 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52391.log.gz 2024-09-05T13:27:15.028 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60979.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60979.log.gz 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62114.log 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37116.log.gz 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44439.log 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61694.log 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62114.log.gz 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55247.log 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44439.log.gz 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61694.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.61694.log.gz 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.51551.log 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60895.log 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55247.log.gz 2024-09-05T13:27:15.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62703.log 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51551.log.gz 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44649.log 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60895.log.gz 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68375.log 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62703.log.gz 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53357.log 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44649.log.gz 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52769.log 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68375.log.gz 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50627.log 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53357.log.gz 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54953.log 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52769.log.gz 2024-09-05T13:27:15.030 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59294.log 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50627.log.gz 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47897.log 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54953.log.gz 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57979.log 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59294.log.gz 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40395.log 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47897.log.gz 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52223.log 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57979.log.gz 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44733.log 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40395.log.gz 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42026.log 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52223.log.gz 2024-09-05T13:27:15.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71023.log 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44733.log.gz 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37961.log 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42026.log.gz 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45741.log 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71023.log.gz 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69510.log 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37961.log.gz 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64467.log 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45741.log.gz 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69510.log.gz 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68585.log 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63963.log 2024-09-05T13:27:15.032 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64467.log.gz 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68585.log.gz 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73375.log 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63963.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63963.log.gz 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72325.log 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73375.log.gz 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53903.log 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40506.log 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72325.log.gz 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66065.log 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53903.log.gz 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43687.log 2024-09-05T13:27:15.033 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40506.log.gz 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73964.log 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66065.log.gz 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50123.log 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43687.log.gz 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67787.log 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73964.log.gz 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53735.log 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50123.log.gz 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52349.log 2024-09-05T13:27:15.034 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67787.log.gz 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72619.log 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53735.log.gz 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44607.log 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52349.log.gz 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45922.log 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72619.log.gz 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42068.log 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44607.log.gz 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60727.log 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45922.log.gz 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42068.log.gz 2024-09-05T13:27:15.035 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39423.log 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60727.log.gz 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72409.log 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41648.log 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39423.log.gz 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72703.log 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72409.log.gz 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52307.log 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41648.log.gz 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70981.log 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72703.log.gz 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52307.log.gz 2024-09-05T13:27:15.036 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41962.log 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65897.log 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70981.log.gz 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41962.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.55835.log 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41962.log.gz 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52265.log 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65897.log.gz 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.55835.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63753.log 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55835.log.gz 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69846.log 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52265.log.gz 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67115.log 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63753.log.gz 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56549.log 2024-09-05T13:27:15.037 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69846.log.gz 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69384.log 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67115.log.gz 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66695.log 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56549.log.gz 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50837.log 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69384.log.gz 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38227.log 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66695.log.gz 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72745.log 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50837.log.gz 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69048.log 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38227.log.gz 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-09-05T13:27:15.038 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72745.log.gz 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51047.log 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69048.log.gz 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72493.log 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51047.log.gz 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72031.log 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72493.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72493.log.gz 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71233.log 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72031.log.gz 2024-09-05T13:27:15.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72451.log 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43917.log 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71233.log.gz 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72955.log 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72451.log.gz 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43917.log.gz 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58411.log 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59336.log 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72955.log.gz 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71149.log 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58411.log.gz 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49073.log 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59336.log.gz 2024-09-05T13:27:15.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52139.log 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71149.log.gz 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59966.log 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49073.log.gz 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51929.log 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52139.log.gz 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41709.log 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59966.log.gz 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70056.log 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51929.log.gz 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74048.log 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41709.log.gz 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67325.log 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70056.log.gz 2024-09-05T13:27:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56129.log 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74048.log.gz 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67325.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67325.log.gz 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37493.log 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58495.log 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56129.log.gz 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62661.log 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37493.log.gz 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65181.log 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58495.log.gz 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53231.log 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62661.log.gz 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50669.log 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65181.log.gz 2024-09-05T13:27:15.042 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39016.log 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53231.log.gz 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74216.log 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50669.log.gz 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38720.log 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39016.log.gz 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.74216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63165.log 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74216.log.gz 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70771.log 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38720.log.gz 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38794.log 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63165.log.gz 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48359.log 2024-09-05T13:27:15.043 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70771.log.gz 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59882.log 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38794.log.gz 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62787.log 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48359.log.gz 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59882.log.gz 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72199.log 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64719.log 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62787.log.gz 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63543.log 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72199.log.gz 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52601.log 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64719.log.gz 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56759.log 2024-09-05T13:27:15.044 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63543.log.gz 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50753.log 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52601.log.gz 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44691.log 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56759.log.gz 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52517.log 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50753.log.gz 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71401.log 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44691.log.gz 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66359.log 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52517.log.gz 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50039.log 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71401.log.gz 2024-09-05T13:27:15.045 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54491.log 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66359.log.gz 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53777.log 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50039.log.gz 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74424.log 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54491.log.gz 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53777.log.gz 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66149.log 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.74424.log: --verbose -- /var/log/ceph/ceph-client.admin.38143.log 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74424.log.gz 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54995.log 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66149.log.gz 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48569.log 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38143.log.gz 2024-09-05T13:27:15.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72871.log 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54995.log.gz 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60349.log 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48569.log.gz 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46866.log 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72871.log.gz 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38144.log 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60349.log.gz 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59504.log 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46866.log.gz 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65812.log 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38144.log.gz 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38905.log 2024-09-05T13:27:15.047 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59504.log.gz 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57431.log 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65812.log.gz 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44985.log 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38905.log.gz 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50585.log 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57431.log.gz 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.44985.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46588.log 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44985.log.gz 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37580.log 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50585.log.gz 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46477.log 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46588.log.gz 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71821.log 2024-09-05T13:27:15.048 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37580.log.gz 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49913.log 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46477.log.gz 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44523.log 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71821.log.gz 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45279.log 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49913.log.gz 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70855.log 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44523.log.gz 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45279.log.gz 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59924.log 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42152.log 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70855.log.gz 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60811.log 2024-09-05T13:27:15.049 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59924.log.gz 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63207.log 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42152.log.gz 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60811.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60811.log.gz -5 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53987.log 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66401.log 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63207.log.gz 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53105.log 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53987.log.gz 2024-09-05T13:27:15.050 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66401.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.67619.log 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66401.log.gz 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63879.log 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53105.log.gz 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50963.log 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67619.log.gz 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63879.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.63879.log.gz -5 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32245.log 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50963.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.50963.log.gz /var/log/ceph/ceph-client.admin.55079.log 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68249.log 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32245.log.gz 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-09-05T13:27:15.051 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55079.log.gz 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56885.log 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68249.log.gz 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52055.log 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50081.log 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56885.log.gz 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73333.log 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52055.log.gz 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37619.log 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50081.log.gz 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64383.log 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73333.log.gz 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45551.log 2024-09-05T13:27:15.052 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37619.log.gz 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56969.log 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64383.log.gz 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61568.log 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45551.log.gz 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52559.log 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56969.log.gz 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64551.log 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61568.log.gz 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60937.log 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52559.log.gz 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64551.log.gz 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65055.log 2024-09-05T13:27:15.053 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60937.log.gz 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59210.log 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44817.log 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65055.log.gz 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62030.log 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59210.log.gz 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32620.log 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44817.log.gz 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57559.log 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62030.log.gz 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38460.log 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32620.log.gz 2024-09-05T13:27:15.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37370.log 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57559.log.gz 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39988.log 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38460.log.gz 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52013.log 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37370.log.gz 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72073.log 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39988.log.gz 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55625.log 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52013.log.gz 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.72073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73922.log 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72073.log.gz 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55625.log.gz 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-09-05T13:27:15.055 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.73922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57263.log 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73922.log.gz 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49031.log 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38068.log 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57263.log.gz 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66527.log 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49031.log.gz 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37202.log 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38068.log.gz 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51593.log 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66527.log.gz 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36605.log 2024-09-05T13:27:15.056 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37202.log.gz 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69174.log 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51593.log.gz 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36605.log.gz 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43645.log 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61400.log 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69174.log.gz 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37115.log 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43645.log.gz 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.61400.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39386.log 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61400.log.gz 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36950.log 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37115.log.gz 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48905.log 2024-09-05T13:27:15.057 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39386.log.gz 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43329.log 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36950.log.gz 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55205.log 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48905.log.gz 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66233.log 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43329.log.gz 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56507.log 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55205.log.gz 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56003.log 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66233.log.gz 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71947.log 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56507.log.gz 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56003.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.56003.log.gz /var/log/ceph/ceph-client.admin.51089.log 2024-09-05T13:27:15.058 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43413.log 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71947.log.gz 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46292.log 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51089.log.gz 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54029.log 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43413.log.gz 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69972.log 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46292.log.gz 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67535.log 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54029.log.gz 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69972.log.gz 2024-09-05T13:27:15.059 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67535.log.gz 2024-09-05T13:27:15.904 INFO:teuthology.orchestra.run.smithi086.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-05T13:27:16.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-05T13:27:18.209 INFO:teuthology.orchestra.run.smithi086.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-05T13:27:18.472 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-05T13:27:18.645 INFO:teuthology.orchestra.run.smithi086.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-05T13:35:00.230 INFO:teuthology.orchestra.run.smithi086.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-05T13:40:06.546 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-05T13:41:36.283 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-05T13:41:36.284 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-09-05T13:41:36.284 INFO:teuthology.orchestra.run.smithi086.stderr:real 14m21.658s 2024-09-05T13:41:36.285 INFO:teuthology.orchestra.run.smithi086.stderr:user 6m24.207s 2024-09-05T13:41:36.285 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m17.353s 2024-09-05T13:41:36.285 INFO:tasks.ceph:Archiving logs... 2024-09-05T13:41:36.286 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478/remote/smithi086/log 2024-09-05T13:41:36.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T13:42:46.757 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-05T13:42:46.769 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-05T13:42:46.770 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T13:42:46.831 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-05T13:42:46.841 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-05T13:42:46.841 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T13:42:46.863 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-09-05T13:42:46.906 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T13:42:46.906 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-09-05T13:42:46.906 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +517us[ +519us] +/- 54ms 2024-09-05T13:42:46.906 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -458us[ -458us] +/- 67ms 2024-09-05T13:42:46.906 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -259us[ -259us] +/- 62ms 2024-09-05T13:42:46.907 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T13:42:46.907 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-05T13:42:46.917 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-05T13:42:46.917 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-05T13:42:46.939 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T13:42:47.092 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-05T13:42:47.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:42:47.123 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-05T13:42:47.134 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-05T13:42:47.160 INFO:teuthology.task.internal:Duration was 2274.396504 seconds 2024-09-05T13:42:47.160 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-05T13:42:47.168 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-05T13:42:47.169 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T13:42:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T13:42:47.295 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-05T13:42:47.295 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-09-05T13:42:47.295 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T13:42:47.316 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-05T13:42:47.316 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T13:42:47.383 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-05T13:42:47.383 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T13:42:47.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-05T13:42:47.571 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-05T13:42:47.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T13:42:47.603 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-05T13:42:47.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T13:42:47.678 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-09-05T13:42:47.700 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T13:42:47.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T13:42:47.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-05T13:42:47.768 INFO:teuthology.task.internal:Transferring archived files... 2024-09-05T13:42:47.769 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-05_12:54:40-rados-wip-bharath4-testing-2024-09-02-0957-distro-default-smithi/7890478/remote/smithi086 2024-09-05T13:42:47.769 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T13:42:47.831 INFO:teuthology.task.internal:Removing archive directory... 2024-09-05T13:42:47.831 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T13:42:47.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-05T13:42:47.891 INFO:teuthology.task.internal:Not uploading archives. 2024-09-05T13:42:47.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-05T13:42:47.917 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-05T13:42:47.918 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T13:42:47.936 INFO:teuthology.orchestra.run.smithi086.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 13:42 /home/ubuntu/cephtest 2024-09-05T13:42:47.936 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-05T13:42:47.946 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-05T13:42:47.971 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-bitmap} rados supported-random-distro$/{centos_latest}} duration: 2274.396504163742 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-05T13:42:47.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T13:42:48.051 INFO:teuthology.run:pass